Deep dive into threading, blocking, sync/async, scheduler internals (split from scztt quarks thread)

Oh, I see now, it’s because any yield “pops back” across any number of stack frames. So if the code invoked from the REPL line called anything that yields, then that would be printed. This interdiction comes down to sclang being less thread safe than usually though.

Are Exceptions also implemented with yield?

I’ve been exploring ways to make receiving (i.e. waiting for) synchronous OSC less kludgy, but the funny thing is that Main.recvOSCmessage which is called directly from C++, is actually called in the context of the Process.mainThread, which is exactly the same as the REPL thread. So, it seems you cannot receive OSC while the main REPL thread is doing something, and also why it would probably be unsafe to hack it to make it be able to yield.


I found the correct idiom for this, but it’s somewhat obscure. Basically, what you have to do is use FlowVar, as if you were using a Ref. Nearly the same code:

	*refGet {
		arg node, ignore=[];
		var server, result = Ref.new;
		node = node ?? { RootNode(Server.default) };
		server = node.server;

		OSCFunc({
			arg msg;
			var snapshot, parsed;
			if (dump) { msg.postln };
			snapshot = TreeSnapshot(server, msg);
			defer {	result.value = snapshot; };
		}, '/g_queryTree.reply').oneShot;
		server.sendMsg("/g_queryTree", node.nodeID, 1);
		^result
	}


	*flowGet {
		arg node, ignore=[];
		var server, flv = FlowVar.new;
		node = node ?? { RootNode(Server.default) };
		server = node.server;

		OSCFunc({
			arg msg;
			var snapshot, parsed;
			if (dump) { msg.postln };
			snapshot = TreeSnapshot(server, msg);
			defer {
				flv.value = snapshot;
			};
		}, '/g_queryTree.reply').oneShot;
		server.sendMsg("/g_queryTree", node.nodeID, 1);
		^flv;
	}

If used directly from REPL the above two behave the same:

t = TreeSnapshot.refGet // -> `(nil)
// By the time you type the next cmd it's updated of course
t.value // -> TreeSnapshot  + Group: 0

// Likewise
t = TreeSnapshot.flowGet // -> a FlowVar
t.value // -> TreeSnapshot  + Group: 0

But if used from a forked thread, the first one (using Ref) has a race of course, while the 2nd one doesn’t.

fork { t = TreeSnapshot.refGet.value }; 
t // nil

fork { t = TreeSnapshot.flowGet.value }
t // -> TreeSnapshot  + Group: 0

And finally, why does flowGet not bomb when you call .value on it in REPL?

The reason is simple: it doesn’t .yield if the value has already been set, which happens in the time it takes to type the next cmd:

FlowVar { // ...
	value {
		condition.wait
		^value
	}
}

Condition { // ...
	wait {
		if (test.value.not, {
			waitingThreads = waitingThreads.add(thisThread.threadPlayer);
			\hang.yield;
		});
	}
}

Morning coffee stuff. But of course, if you actually try in REPL

t = TreeSnapshot.flowGet.value

it still ERRORs. The amazing usability of SC.

So I’m not sure that receiving OSC messages in the context of Process.mainThread, which is how it’s currently done, was a particularly smart decision. For clarity, this a sclang-wide problem, it’s particular to this quark.

The almost funny part is that Thread has a terminalValue field, which is used to store the result for alwaysYield. But the problem is that there’s no way to join a Thread to another in sclang, i.e. wait for it to finish and get the return value, even though there’s a slot for that in Thread. (By the way, the whole CondVar shebang does nothing to improve on this problem. It addresses something else.)

Even if I hacked some thread join, it still wouldn’t work for OSC because the C++ function that gets the latter (localServerReplyFunc) wants to acquire the global interpreter lock. So from the REPL it’s impossible to get the OSC in one command. You need two, so that interpreter lock is released in-between and the OSC can actually go through the interpreter. So we’re stuck with the explicit continuation-passing-style for this and OSC in general.