I’m trying to display images in sync with audio and having issues … has anyone had any luck getting tight sync between displayed images and server audio?
likewise wondering is anyone has benchmarked the discrepancy between AppClock and SystemClock? On my system they pull apart by maybe a half second per minute!
one thought is that I might be better off using systemCmd and some external tools though this would be sad…
any insights appreciated!
What sort of images/audio are you using and how are you going about getting them to sync? I get very tight sync with server sending position values over osc…
What I’m doing is pretty primitive I expect!
I have a class that schedules patterns, events and functions on the system clock (bind-ing the functions to adjust for latency).
I’m displaying .png images in a Qt window
using this method to display the images
plot{|markerName monitor=(-1)|
var image=this.mark(markerName);
var w = Window(bounds:Rect(1500*monitor,200,1400,800),border:false)
.background_( Color.black)
.drawFunc_({Pen.drawImage(Point(100,100),image,operation:'sourceOver',opacity:1)})
.front; //}.defer(Server.default.latency);
^w
}
AppClock has never been guaranteed to stay in sync with SystemClock or TempoClock.
The way to handle that is to sequence the graphics timing on the same clock that is handling musical rhythm, and defer each isolated graphics update.
(
c = Color.black;
d = 100;
u = UserView(nil, Rect(800, 200, 400, 400)).front
.drawFunc_({ |view|
var radius = d.linlin(-7, 14, 150, 30);
Pen.color_(c)
.fillOval(Rect.aboutPoint(view.bounds.extent * 0.5, radius, radius));
});
)
(
p = Pbind(
\degree, Pwhite(-7, 14, inf),
\dur, Pexprand(0.25, 1.2, inf),
\graphics, Pfunc { |ev|
d = ev[\degree];
c = Color.rand;
{ u.refresh }.defer(s.latency);
}
).play;
)
p.stop;
I’m using patterns because I like them, but you can do the same thing in a Task or Routine.
The \graphics Pfunc activates exactly in time with the notes. Only the graphics update (refresh) gets pushed off to AppClock by defer.
hjh
Thanks for this clear example!
… I think one of the problems I have been having is that I have been trying to create the window in sync with the sounds and the window creation may add some non-trivial delay…
I do wish didn’t have to manually defer and adjust for system latency though!
A bit ironic, because the first sentence explains why the second sentence is necessary 
Graphics operations are not guaranteed to be fast, so they have to be on a lower priority thread.
There was a discussion once whether graphics methods might automatically defer, but it’s easy to imagine cases where you need A, B and C to happen in that order. An obvious case would be Button.new.states_([["OK"]]) – the button must exist before you can set its display strings. If new and states_ are separately deferred, we would have to be very sure that it will never try to handle states_ first – i.e. a lot of careful design and testing. (And, tbh, there hasn’t been that much demand for it.)
Latency… the graphics updates don’t know how much latency is applied to server messaging. If you’re using patterns and events, server .latency is a good guess but you might be using a custom event prototype that handles latency differently. In a Routine, latency is totally up to you. So it’s another one that seems like it should be easy but when you try to pin down what the latency is in every possible situation (and if it’s automatic, then it does have to handle every foreseeable case), then it’s not so straightforward.
I agree that the syntax looks clunky but improving that is a bigger job than it would seem. (But, SC is a do-ocracy so anyone who feels strongly about it can start by building some tests to see if auto-scheduling on AppClock is order-safe or not.)
hjh