Executing async code within a Routine (for Language Server)

Music Code Tennis
1

Okay, time for me to ask a question :slight_smile:

I’ve been trying to pin down something with the Language Server, and I can’t quite get it right. Here’s the challenge:

I’d LIKE to have an architecture where code that you execute with e.g. Cmd+Enter is executed in a Routine - this makes it easier to control the context of execution for things like the current environment, but I was also hoping it would allow you to properly run asynchronous code that e.g. waits for a condition (in ScIDE, you would get a “cannot yield from this thread” error in this case). I’ve been trying something like this (a little simplified):

        handlerThread = Routine({
            |f|
            Environment().push; // root environment is persistent per-thread
            inf.do {
                f = f.value.yield; 
            }
        });

Then, I can do:

function = "someCode".compile;
handlerThread.next({ 
    thisProcess.nowExecutingPath = document.path;
    function.value().postln;
});

The problem is, of course, that if I execute code like:

"one".postln;
1.wait;
"two".postln;

… then the 1.wait yields to my handlerThread, which basically ends execution. Then, the NEXT code I execute picks up after my 1.wait (giving me “two”) BUT the new code I wanted to execute is dropped (my handlerRoutine.next just ends up getting passed through the 1.wait and it’s not caught).

Beyond this, I’m already seeing weird crashes during e.g. some cases where an error is thrown - I’m assuming I’m hitting a particularly complicated case of threads-executing-threads with several layers of Exception handling. I’d like to ofc fix the crashes, but until that can be fixed - I’m taking this as a warning that … what I’m doing might be a little over-complicated? I don’t want the most basic thing you can do - execute code - to be built on top of something shaky.

So, here’s the challenge:
I want a Routine where I can inject other functions - which can contain async code! - to be run. I want to run these functions until they are finished, and then return the result. Errors should be caught in a try-catch (or via the thread error handler) for reporting. I’d also like SOME control over the async execution, insofar as it would be nice to give users the option of — (1) wait for async code to complete before executing more, or (2) don’t wait, and allow multiple executions of async code to happen at the same time (with the condition that you might get out-of-order return values). I’m curious what other people can come up with for this - everything I’ve tried this afternoon has felt too complex, and I feel like I might be missing something more elegant?

2

Maybe use a Condition and a fork like this:

(
d = ();
d.handlerThread = Routine({
	|f|
	Environment().push; // root environment is persistent per-thread
	inf.do {
		c = Condition(false);
		fork{ // fork with condition
			f = f.value.yield;
			c.test = true;
			c.signal;
		};
		c.wait;
	}
});
d.function = {
	"one".postln;
	1.wait;
	"two".postln;
};
d.handlerThread.next({
	d.function.value().postln;
});
)

Best,
Paul