I understand the change in philosophy from sc2 to sc3, but is it now impossible do to “front end” synths that spawn other synths without involving the/a client? (I can see sending OSC-whatever from server to client which in response starts new server nodes via s_new directly or its many client-side wrappers). But is it impossible to do that in sc3 without involving the/a client?
Basically I’m looking for some SC3 server-side-way to do the Csound-equivalent of recursive UDOs.
You absolutely need a client, no exceptions.
It is possible, however, to pre-allocate synths that will be triggered later by server-side triggers. I can also see in my head how you could incrementally pre-allocate synths – a server-side trigger would both activate the next pre-allocated synth, and send a message to the client saying that it should preallocate a few more synths.
hjh
Not if the number of these synths can be arbitrary (as in hundreds new ones per minute). I suppose one can write a UGen that fires s_new though.
The pre-allocator could create multiple nodes at once and store node IDs in a Buffer used as a queue. Even if you needed a hundred nodes per second, what I’m suggesting can be designed in a way that would scale up.
It’s all possible – that’s not to say it’s easy, only to say that there are ways to do it.
The problem for a server-side s_new is how to represent control names in the server. UGen inputs don’t support strings. Poll and SendReply use a workaround that’s designed for very short strings but that won’t scale up to dozens of arguments. For me, the C++ changes that would be required to make this convenient would be harder than a pre-allocation approach (but maybe a C guru would feel differently).
hjh
UGen inputs don’t support strings.
Unit commands do, but they are very awkward to work with… And they have a nasty bug which I still didn’t have time to fix (currently, I just use a hacky workaround)
PollandSendReplyuse a workaround that’s designed for very short strings but that won’t scale up to dozens of arguments.
There’s also the API method NodeReply, but it only allows a single string argument (cmd name) followed by an array of float values. I wanted to propose a new API method which allows to send arbitrary OSC messages back to the client. This would be handy for UGens like VSTPlugin, which need to send some state back to the Client(s). Currently, I’m encoding string arguments as “binary” float arrays… Another way would be to stuff both the actual cmd name and arguments into a single string. Both are equally awkward…
Finally, there is a way to send arbitrary OSC messages/bundles from the Server to itself:
a UGen can call DoAsynchronousCommand with a completion messages, which will be performed after stage3 has been completed successfully. Note that a missing stage function also counts for success, so you can simply write:
// arguments are: World* inWorld, ReplyAddress* inReplyAddress, const char* cmdName, void* cmdData,
// AsyncStageFn stage2, AsyncStageFn stage3, AsyncStageFn stage4, AsyncFreeFn cleanup,
// int completionMsgSize, void* completionMsgData
DoAsynchronousCommand(inWorld, 0, 0, 0, 0, 0, 0, 0, msgSize, msgData);`
By not providing cmdName you effectively surpress the sending of a /done message to the Client.
supercollider/common/scsynthsend.h contains a small library for creating OSC messages/bundles.
Of course, this is a ugly hack 
I’ve experimented with it, but eventually didn’t need it.
Note that abusing DoAsynchronousCommand introduces a non-deterministic delay because we have to go to the NRT thread and back to the RT thread. It’s totally possible for the plugin API to contain a method for immediately sending a Server message from within a UGen. This could enable some interesting things. On the other hand, this raises the question how to keep Server and Client(s) in sync. Currently, the clients are responsible for “allocating” Synth and Buffer IDs. If you create a Synth or Buffer on the Server, how should you keep track of IDs and prevent collisions with the Clients?
Using Server commands just for controlling Synths might be less controversial, but I can’t think of a real use case right now…
Yeah, there’s server-client allocator sync issue to consider… on top of the other issues.
Thanks for the detailed analysis on the roadblocks for this.
Sorry to bring the topic up, but I was intrigued by this statement. Do you have an example of this behaviour?
I didn’t, but hacked one up just now.
This works to trigger nodes that are already set up.
I didn’t find a way to re-trigger an envelope on a node that had run once, paused itself and then needed to be reawakened.
Also, I can’t find a way for one server node to address specific control inputs of a different server node. (You could conceivably map those control inputs to control buses, and the controller node could Out.kr or ReplaceOut.kr to set the value, but I didn’t try to do that today. I don’t expect I will try it later either, tbh.)
So, this example creates 10 paused nodes, and the c controller synth wakes them up one by one, following a trigger in the server. You could replace the Dust.kr by an Onsets.kr or other analytical process to have instant response to audio triggers.
(
SynthDef(\defaultP, { arg out=0, freq=440, amp=0.1, pan=0, time = 2;
var z;
z = LPF.ar(
Mix.new(VarSaw.ar(freq + [0, Rand(-0.4,0.0), Rand(0.0,0.4)], 0, 0.3, 0.3)),
XLine.kr(Rand(4000,5000), Rand(2500,3200), 1)
) * Linen.kr(Trig1.kr(Impulse.kr(0), time), 0.01, 0.7, 0.3, doneAction: Done.freeSelf);
OffsetOut.ar(out, Pan2.ar(z, pan, amp));
}, [\ir]).add;
)
(
a = [-7, -1, 0, 2, 1, 5, 4, 8, 5, 6].collect { |degree|
Synth.newPaused(\defaultP, [freq: (degree.degreeToKey(Scale.major) + 60).midicps]);
};
b = Buffer.sendCollection(s, a.collect(_.nodeID), 1);
~dummy = { Silent.ar(1) }.play;
)
(
c = { |bufnum, neutralID|
var trig = Dust.kr(1.5); // here's the server-side trigger
var num = BufFrames.kr(bufnum);
var idSource = Demand.kr(trig, 0, Dbufrd(bufnum, Dseries(0, 1, num), loop: 0));
var stop = FreeSelfWhenDone.kr(idSource);
var id = Select.kr((trig > 0), [neutralID, idSource]);
[trig, id].poll(trig); // can remove, just for display
Pause.kr(trig, id);
}.play(outbus: 1000, args: [bufnum: b, neutralID: ~dummy.nodeID]);
)
~dummy.free; b.free;
// in case something broke, stop the others here
a.do(_.free); c.free;
The ~dummy node is necessary because Pause will pause that node ID when trig becomes 0. In early tests, I found it was easy to pause node 0, in which case, everything dies. So we give it something else to pause without doing any damage.
If you want the process to run indefinitely, you could have c do a SendReply to have the client generate more paused nodes and update part of the buffer that isn’t being used at that moment. (Also delete var stop.) I’m afraid I won’t have time to work that out for you, though. This is as far as I can take this for now.
hjh
Thank you very much, the example is super clear!
One last question: is there a way to make the triggering (Pause.kr) run at audio rate? Or is it impossible due to the fact that setting a Node to run needs to be done at the start of each audio buffer, thus at control rate?
Nope.
There is also no way to do s_new from the client in the middle of a control block. (OffsetOut can offset the audio, but the synth still runs on control block boundaries.)
hjh
However, I guess in the case of using Pause.kr, no OSC timestamp is provided, making OffsetOut behave exactly the same as Out?
The timestamp could be provided at the time of creating the paused node, but not at trigger time.
If you need sample accurate triggering, you’d have to start the node not paused, and trigger the envelope from an audio rate signal.
You might not even need sample accurate triggering though. This is one of those things where it’s easy to think “we should be able to do that” but the engineering effort outweighs the real world benefits.
hjh
I have been thinking today of ways of implementing this. The idea I came up with is to have a PolySynth class that would work just as Synth, but it would pre-allocate a specific number of unique IDs. For example:
a = PolySynth(\default, n: 128); //Pre-allocate 128 IDs. Both Client and Server will know about them.
//Client interface
a.newSynth; //Create a new Synth, it will use the first ID available. The creation happens on the server at the next buffer cycle!
a.newSynth; //Create a new Synth, it will use the second ID available
//After the first Synth has been freed with DoneAction or whatever
a.newSynth; //Create a new Synth, it will use the same ID as first one, as that one has been freed
//Audio rate interface
b = {Poly.ar(a, Impulse.ar(1))}.play //First argument: a PolySynth. Second argument: audio triggers
The voice scheduling, this way, would entirely be executed on the server, also allowing for audio rate control of the creation of the new nodes, perhaps with a trigger interface (1 == create new Synth).
I know this is still all quite sketchy, but I think it could be powerful for certain audio rate patterning happening entirely on the server. To schedule new synths in between an audio block, the same approach as OffsetOut can be taken, where the Node would actually be created at the start of the block, but the output would be delayed accordingly.
Of course, the big drawback of this approach would be the limit of n concurrent Synths playing at one time, but having big numbers for n for specific processes, like granular synthesis, is relatively inexpensive, as it just about keeping a list of IDs alive in both Client and Server.
What do you think?
It might work if the Pause.kr synth is earlier in the node tree – then, a trigger in the middle of a control block would change the target node’s state before node evaluation gets around to that node. It would even be possible to set the offset for OffsetOut (I guess…? I’m not familiar with that part of the code).
The trickiest thing is, I think this approach would really need a way to set controls at trigger time. I don’t have an idea for that, but it was the biggest problem I faced when hacking up that little prototype.
hjh
Yeah that’s the tricky bit I’m trying to wrap my head around. Perhaps through using a combination of Buffer writing and Demand rate stuff, but I haven’t tried yet.
Yeah, apparently what happens is that they hardcode the strings as ASCII floats in the graph, prepended by the string length. (That’s Pascal format, apparently.) E.g. “Hello world” is [11, 72, 101, 108, 108, 111, 32, 119, 111, 114, 108, 100].
/*
d = SynthDef(\asdf, { Poll.kr(0, 0, "Hello world") })
d.dumpUGens
[ 0_Impulse, control, [ 0, 0 ] ]
[ 1_Poll, control, [ 0_Impulse, 0, -1, 11, 72, 101, 108, 108, 111, 32, 119, 111, 114, 108, 100 ] ]
*/
The stranger thing is that if I turn on debugOSC, I don’t see any strings sent at all,
so I’m guessing there’s some kind of refencing used later
-> Synth('asdf' : 1031)
[ 9, "asdf", 1031, 0, 1, 0 ]
Hello world: 0
Could you briefly explain that that stage3 means or where can I read about those stages?
That’s a fair point but there was a discussion here about local buffers created on the server that apparently have conflicting numbers. So it’s not like the server doesn’t create any of those “by itself” at all.
I’m confused in this example what is actually making the nodes un-pause themselves. Your are starting them as newPaused… and then you’re pausing them again with Pause.kr, it seems, but how to they start going in sequence, I don’t quite understand since the internal gate of each note Linen.kr(Trig1.kr(Impulse.kr(0), time) goes off immediately if a synth like that is started normally (not paused). So how do the nodes go from the newPaused state to playing? According to the help for newPaused, one way to do that is to call run on the node, but I don’t see your code do that anywhere, yet the example works. SC can be so confusing 
That’s not quite correct. LocalBufs are not Buffers.
zalloc()).RTAlloc()).The trick here is found in the Pause help file: “gate – When gate is 0, node is paused, when 1 it runs.”
trig is nonzero for exactly one control cycle. At that exact moment, id is allowed to be one of the nodes – Pause.kr with a nonzero gate unpauses.
Then, in the next control cycle, trig returns to 0, and the Select UGen switches id away from the node that you want to run. So the “real” node keeps going, and the dummy is (re-)paused.
hjh
Ah, ok, now it makes sense… Although the doc is a bit confusing as it first says just “When triggered, pauses a node.” Only later it adds “When gate is 0, node is paused, when 1 it runs.” I was able to run them manually (e.g. as in Node.basicNew(s, 1092).run) after they were created so they were clearly paused at that point. I could find any way to list that status though, e.g. neither query on the node nor s.queryAllNodes tells you if node is running or paused…
So, does that mean that Pause.kr cannot re-awaken a node that had called PauseSelfWhenDone? The help for the latter is in “FIXME” stage, literally.