Syncing OSC messages to multiple JS clients from pattern by sending OSC in advance with timestamp

Questions
#1

Dear All,

I am working on a visual metronome with Open Stage Control (JS) and would like to sync multiple clients. I provide a basic example below within SC. Is it possible to introduce a delay for the latency and send the sendPattern ahead of time with a future timestamp so that the client can know when to actually trigger the action? Thanks in advance. Suggestions welcome for other ways to potentially do this.

// run this to receive the messages
(
OSCdef(\receiver, {arg msg, time, addr, port;
	msg.postln;
}, \playing);
)

// run this to send the messages
(
var playPattern, sendPattern, stream, offset, offsetStream, tempoClock, player;

Event.addEventType(\osc, {
	if (~addr.postln.notNil) {
		~addr.sendMsg(~path, *~msg);
	};
});

playPattern = Pbind(
	\freq, Pseq([300, 350, 400, 450], inf),
	\dur, 1
);

sendPattern = Pbind(
	\type, \osc,
	\addr, NetAddr.localAddr,
	\path, "/playing",
	\msg, Pseq([1, 2, 3, 4, 5, 6, 7, 8], inf),
	\dur, 1
);

stream = Ppar([playPattern, sendPattern]).asStream;
offset = stream.fastForward(2);

offsetStream = Routine({offset.wait});

player = EventStreamPlayer(offsetStream ++ stream);
tempoClock = TempoClock(45/60.0);
player.play(tempoClock);
)
#2

sendBundle can do timestamps. sendMsg can’t.

n = NetAddr("127.0.0.1", 7878);

n.sendMsg(\hello, 123);            // immediate

n.sendBundle(0.2, [\hello, 123]);  // timestamped for 200 ms later

hjh

#3

Thanks James,

On my machine sendBundle seems to send immediately but adds the timeStamp as an offset in milliseconds to the time arg in the receiver. See below how I tested that. But that still does not seem to address my problem. What I really need (I think) is to delay the pattern by some offset to account for latency and then send in advance the message with the timestamp for which the event is scheduled. That way, I can pass the message ahead of time to the JS client with the absolute time in which the event will be scheduled in SC. So I guess the question is whether or not there is a way to get from the SC scheduler, the absolute time that the event will occur in the future. I hope that makes sense.

// run this to receive the messages
(
OSCdef(\receiver, {arg msg, time, addr, port;
	[msg, time].postln;
}, \playing);
)

// run this to send the messages
(
var playPattern, sendPattern, stream, offset, offsetStream, tempoClock, player;

Event.addEventType(\osc, {
	if (~addr.postln.notNil) {
		~addr.sendMsg(~path, *~msg);
		~addr.sendBundle(1000, [~path, Date.gmtime]);
	};
});


playPattern = Pbind(
	\freq, Pseq([Rest(), 300, 350, 400, 450], inf),
	\dur, 1
);

sendPattern = Pbind(
	\type, \osc,
	\addr, NetAddr.localAddr,
	\path, "/playing",
	\msg, Pseq([1, 2, 3, 4, 5, 6, 7, 8], inf),
	\dur, 1
);

stream = Ppar([playPattern, sendPattern]).asStream;
offset = stream.fastForward(0);

offsetStream = Routine({offset.wait});

player = EventStreamPlayer(offsetStream ++ stream);
tempoClock = TempoClock(45/60.0);
player.play(tempoClock);
)
#4

What confused me here is “delay the pattern” (implying that the message send/receive/action should be more or less synchronized) and “send in advance” (implying that the action is not synchronized). To me, it doesn’t make a difference whether the message is sent 2 ms early, or 200 ms – it’s early, and the receiver will have to account for that either way.

As I understand it, in SC we specify the timestamp in terms of an offset from now, but the OSC timestamp is an absolute time. OSC spec 1.0, time tag semantics:

Time tags are represented by a 64 bit fixed point number. The first 32 bits specify the number of seconds since midnight on January 1, 1900, and the last 32 bits specify fractional parts of a second to a precision of about 200 picoseconds. This is the representation used by Internet NTP timestamps.

SC is (AFAIK) compliant here. What the receiver does with the absolute time in the bundle is not within SC’s control. The receiving app should be able to decode the absolute time but maybe it doesn’t…?

hjh

#5

Again. Thanks. And my apologies if I have been unclear. It is not the osc message timestamp that the JS client needs to know. What I want/need to send is the scheduled time for which the event in the pattern will be played/triggered on the SC server.

#6

It is not the osc message timestamp that the JS client needs to know. What I want/need to send is the scheduled time for which the event in the pattern will be played/triggered on the server.

But aren’t those the same?

I just recently did a project where I had to send timestamped events from SC to the browser. Here’s what I did:

  1. SC sends OSC bundles to a custom python server
  2. the python server receives the OSC bundles, extracts the timestamp and forwards the message with the timestamp to a websocket
  3. the browser client receives the websocket messages and schedules them according to the timestamp
#7

Ah. I see where I was confused. In the OSCDef I was using for testing I did not understand the following from the documentation for the time argument:

the time received (for messages) / the OSC bundle’s timestamp (if the message was in a bundle)

But here I am still confused. In the code below where I send both a message and a bundle, the timestamp from the bundle is not absolute time.

// run this to receive the messages
(
OSCdef.new(\receiver, {arg msg, time, addr, port;
	[msg, time].postln;
}, \playing);
)

// run this to send the messages
(
var playPattern, sendPattern, stream, offset, offsetStream, tempoClock, player;

Event.addEventType(\osc, {
	if (~addr.postln.notNil) {
		~addr.sendMsg(~path, ~msg);
		~addr.sendBundle(1000, [~path, ~msg]);
	};
});


playPattern = Pbind(
	\freq, Pseq([Rest(), 300, 350, 400, 450], inf),
	\dur, 1
);

sendPattern = Pbind(
	\type, \osc,
	\addr, NetAddr.localAddr,
	\path, "/playing",
	\msg, Pseq([1, 2, 3, 4, 5, 6, 7, 8], inf),
	\dur, 1
);

stream = Ppar([playPattern, sendPattern]).asStream;
offset = stream.fastForward(0);

offsetStream = Routine({offset.wait});

player = EventStreamPlayer(offsetStream ++ stream);
tempoClock = TempoClock(45/60.0);
player.play(tempoClock);
)
#8

The absolute time is calculated inside the sclang scheduler. The actual OSC bundle, as received by the other application, will contain the absolute time. However, when sclang receives an OSC bundle, it converts the absolute timestamp back to a relative timestamp.

#9

Thanks. that is good/important to know! I have sent a question to the Open Stage Control developer asking how to unpack the timestamp. I would have potentially figured that out but I have yet to get timestamps in my target client.

So then to account for network latency. I can just add an offset/delay to the “playing pattern” (like a one second Rest() at the beginning or using Ptpar) and then schedule the corresponding event on the client for current timestamp + delay. That will still be negligibly inaccurate but fine for my purposes (I hope).