Yeah I think that goes back to the original addToSynth. I remember being surprised when I first realised that buildSynthDef is a classvar of UGen, but I don’t know what the original rationale was, and it’s changed a lot since then.
Actually this could be better worded. UGens with side-effects need to be correctly ordered relative to UGens which depend on either the new or the previous state. A relevant example is ‘multichannel expanded’ PV UGens. The implicit PV_Copy instances need to run before anything later in the chain modifies the buffer. (Thus width first.)
This is how I’m suggesting this be solved, by inserting ‘weak’ edges.
From
To
This way, PV1 can’t be executed before the copies, but the copies can still be nicely sorted to avoid excessive wire bufs.
So an execution order might be
- Buffer
- PV_Copy1
- PV2
- PV_Copy2
- PV3
- PV1
Whereas strictly evaluating them width first is unnecessary. Technically, the buffer could also be recycled and the copy avoided in certain situations, but that is a little more advanced than needed here (at least for now).
1 Like
That’s also the case after using and adapting graph libraries in Haskell. Both start with edges to manage dependencies and construct the graphs, which is the design used in most implementations.
The main difference is that sc constructs synthesis graphs step-by-step since that’s how the synthdef dsl works. Building a graph explicitly involves clearly defining the nodes and edges in most implementations. I believe most work elsewhere is done this way.
The solution I came up with for side effects was to have a type class or just a simple function with priorities. Elsewhere, it can be used in the topo sort recursive code (go style), for example, sortedSuccessors = sortBy (comparing (priorityFn ...., etc. (of course, type classes allow more fine-tunning)
instance HasSideEffect UGen where
hasSideEffect = \case
PV_Copy{} -> True
PV_MagAbove{} -> True
Out{} -> True
_ -> False
sideEffectPriority :: UGen -> Int
sideEffectPriority ugen = case ugen of
PV1 -> 1
PV_Copy -> 2
Buffer -> 3
_ -> 0
EDIT: I think recycling wire buffers, especially large ones, is usually possible. The benefit of large graphs with 50+ UGens can be major.
EDIT2: This is one of the code pearls: strong components can be discovered “on the fly” during a depth-first search—good correct references: https://logarithmic.net/pfh-files/blog/01208083168/tarjan.py AND https://cs.stanford.edu/~knuth/fasc12a+.pdf#page=10 (dates back to 1972). Besides, graphs are a well-studied problem. Just take a look around and see if there is a feeling about our old code.
One oddity is groups of sequential Ins or Outs, which don’t need to be ordered, but ReplaceOut does. This is perhaps important because without this stipulation, there could be limits on the ordering of multichannel branches that end in an Out.
This sounds like a great and – in hindsight! – obvious solution. Some UGens have implicit interdependencies and expressing these with additional graph edges makes them visible to the graph sorting algorithm.
BTW, some of these dependencies cannot be automatically deduced and need to expressed explicitly by the user. One common example is a BufWr followed by a BufRd:
var sig = SinOsc.ar;
var phase = Phasor.ar(0, 1, 0, BufFrames.kr(buf));
var writer = BufWr.ar(sig, buf, phase);
var reader = BufRd.ar(1, buf, phase - 44100, loop: 1);
(Taken from firstArg question - #2 by jamshark70)
Nothing in the example above guarantees that BufWr is scheduled before BufRd.
Currently, the workaround is to use the (rather obscure) !< operator:
var sig = SinOsc.ar;
var phase = Phasor.ar(0, 1, 0, BufFrames.kr(buf));
var writer = BufWr.ar(sig, buf, phase);
var reader = BufRd.ar(1, buf <! writer, phase - 44100, loop: 1);
This ensures that BufWr runs before BufRd. Note that <! actually creates a BinaryOpUGen with the operator firstArg which just discards the second argument.
I think a better solution would be to have a method that adds a “weak edge” between two UGens, e.g.:
var sig = SinOsc.ar;
var phase = Phasor.ar(0, 1, 0, BufFrames.kr(buf));
var writer = BufWr.ar(sig, buf, phase);
var reader = BufRd.ar(1, buf, phase - 44100, loop: 1).after(writer);
IMO this is much easier to understand than the <! operator. Also, it wouldn’t require an additional UGen.
1 Like
I really feel like adding an extra layer of specification is a mistake here, and should be unnecessary. The user has already explicitly specified intended order. This is straightforward and intuitive. The sorting algorithm should only optimise away from that where it is guaranteed there will be no issue. Yes, you could fix an exceptional case if needed with a method like that, but not violating the principle of least surprise should be a far higher priority than say minimising wirebuf usage in my opinion.
In the vast majority of cases an extra wirebuf is harmless. Better to supply a weird exception or some guidance for super memory or CPU constrained cases if that’s really needed.
Indeed! Similarly, another tricky one might be two unconnected chains of UGens, in the same synth, one with an In and one with an Out. (I don’t have a real example, but I think it’s conceivable.) The sort must not optimise the in chain to be after the out chain, as the Out will mix with what’s already on the bus, creating feedback where it’s not intended.
I really feel like adding an extra layer of specification is a mistake here, and should be unnecessary
I did not propose an extra layer, but an alternative to the existing specification (= the <! operator)
The user has already explicitly specified intended order.
In the first example they didn’t! There is no ordering expressed, neither implicitly nor explicitly. writer and reader are two separate graphs that are not connected by any edges. That’s why we currently need the (rather strange) <! operator, which creates an (artificial) connection between the two graphs.
(Side note: Pd has the same issue. Two subpatches are only guaranteed to be evaluated in order if connected with signals; in some cases, the user needs to make “dummy” connections to force a particular ordering.)
All I’m saying is that if we choose to implement some “weak edges” solution, we could take the chance and provide a more intuitive alternative to the <! operator.
Sorry, they are both an extra layer, which I think is workaround which should be unnecessary. If we’re going to rework this, let’s take the chance to get rid of that.
They absolutely did. 
They clearly expect it to happen in the order it was written, so the BufWr is before the BufRd. If that’s not the case, then how can we even talk about it being sorted ‘wrong’?
My point is, where there’s any possibility that it could go wrong, use the order it was written. It’s confusing to require the user to add additional code to say, “yes, I really meant it.” We should be able to ‘automatically deduce’ the intention from the code, or? Making them reiterate it seems like a failure to me.
3 Likes
We could try to detect graph segments that share no connections and force them to be evaluated in the “written” order. We would then need to apply the topological sort on each graph segment individually.
Personally I believe in “explicit is better than implicit”.
At the end of the day, I’m happy with any solution that makes the <! hack obsolete
2 Likes
If we start from the principle that we differ from user order only where it doesn’t matter we should be able to achieve that.
That may mean an implementation that uses slightly more wirebufs in some cases. I think that would be a price well worth paying, and I suspect very much in keeping with the original design intention. Ease of user comprehension, avoiding exceptions, and minimising surprise are all important considerations.
Scott, I agree with all the principles there.
But SynthDef syntax is designed to be expressive and user-friendly, and that’s good, but this sometimes leads to implicit dependencies and execution order. The SynthDef DSL already requires some interpretation to turn each unit into a corresponding node in the graph. I understand this as part of the design.
The case with BufWr/BufRd seems like a typical situation in which some interpretation will occur. Most UGens have explicit dependencies defined by their connections. These are naturally strong edges in the graph and form the basis for topological sorting. Implicit dependencies (as some examples were mentioned here) should be used judiciously when implicit dependencies cannot be naturally resolved.
In terms of optimization, would be nice to have some numbers.
1 Like
I think we’re basically in agreement.
A wrinkle – now, I’m aware, basically nobody is seriously doing this – but we did have questions in the past about whether it’s always necessary to write calculations in order, and – it’s not strictly necessary.
(Note at the top of the example, freqs depends on detunes but detunes is written second 
)
So there’s an implicit design decision being made here, which rules out an expressive alternative.
hjh
3 Likes
A SynthDef explicit syntax would be something more like this. By the way, it would be a good idea to have a readable intermediate DSL, but unlike SynthDef DSL, ensuring explicit graph construction,
I use an experimental scheme-like and JSON-like version of this idea to exchange between languages and GUI
synthdef cute_synth {
node 1: SinOsc.ar(frequency: 440, phase: 0)
node 2: CutePhasor.ar(start: 0, increment: 1, end: BufFrames.kr(buf))
node 3: BufWr.ar(signal: node 1, buf: buf, phase: node 2)
node 4: BufRd.ar(buf: buf, phase: node 2 - 44100, loop: 1)
edge 1 -> 2
edge 2 -> 3
edge 3 -> 4
executeBefore(node 3, node 4)
output.ar(0, node 4)
}
or
cute_synth :: SynthDef
cute_synth = SynthDef
{ name = "cute_synth"
, nodes = [(1, sinOsc), (2, phasor), (3, bufWr), (4, bufRd)]
, edges = [ Edge 1 2, Edge 2 3, Edge 3 4 ]
, executeBefore = [(3, 4)]
, output = (0, Out 0 bufRd)
}
sinOsc = SinOsc 440 0
phasor = Phasor 0 1 bufFrames
where
bufFrames = 44100
--etc
Microsoft Excel and Haskell are some of the few languages that let you structure your program in a way that allows functions and data types to be declared and used in arbitrary order. (the last example above shows that a little)
It’s a special kind of expressivity; Having a clean syntax with those features would be great.
that’s amazing!
To be clearer though, I guess we should say that the order is clear from the code rather than strictly written. This is also more trivially true in say Saw.ar(SinOsc.ar(1).range(440, 880)) in which the Saw is created but not written before the SinOsc.
I think in this case that principle remains equally true, despite the extensive obsfucation via all those interconnected functions resolving only at the end in a cascade of lazy evaluation. It is nevertheless possible to trace it through and determine the order indicated by the code.
Thinking about processes and sequences of actions may seem more intuitive. Still, if we look at natural languages, the most trivial thing is mixing left-side definitions (like where-clauses) and right-side ones. It’s only possible with this other way (“referential transparency,” as they say). That’s also the way we do algebra. Most computer languages are formally very odd compared to most other things, like how we speak and use algebra.
Of course, you have more freedom in writing your program, so one expects you to use this to clarify it, not obfuscate.
I don’t know how a synthdef like this would feel, but the little dsp code I did this way was not bad.
Related to this topic, Haskell uses graph representation to evaluate expressions and update nodes with their resulting value. All shared expressions are only evaluated once, which is more efficient.
So, GraphDef is halfway there, although the other half is not easy to get right. 
Re the buffer ordering and that scary little !<.
How I’m thinking of handling this is making a UGenConnectionDelegate. A buffer UGenConnectionDelegate would record the last buffer access as the UGens are being added to the graph, and insert these weak edges to any newly added UGen. This works because the added order is a valid topological ordering.
In buffer read and write’s case this is going to be added weak edges when the new ugen is a different type (read or write) from the previous.
There’d be a similar system for bus access and message sending.
Each UGen has a (class)method that lists which UGenConnectionDelegate (or multiple) it needs. This would require an extra step for UGen authors.
// And example of a UGen that reads a buffer and sends message.
connectionDelegates { ^[BufferConnectionDelegate, MessageConnectionDelegate] }
bufferType { ^\read } // requires by the connection delegate
messageType { ^\send } // sim.
For backwards compatibility, we could have a panic mode for when none is specified and insert weak edges to everything.
This is a more granular and verbose that the current pure/non-pure, but it thinks it’s more flexible and explicit. And it hopefully makes that <! operator obsolete.
Side note, I never knew <! existed or was required, so I’ve been writing synthdefs wrong for years.
1 Like