I really feel like adding an extra layer of specification is a mistake here, and should be unnecessary
I did not propose an extra layer, but an alternative to the existing specification (= the <! operator)
The user has already explicitly specified intended order.
In the first example they didn’t! There is no ordering expressed, neither implicitly nor explicitly. writer and reader are two separate graphs that are not connected by any edges. That’s why we currently need the (rather strange) <! operator, which creates an (artificial) connection between the two graphs.
(Side note: Pd has the same issue. Two subpatches are only guaranteed to be evaluated in order if connected with signals; in some cases, the user needs to make “dummy” connections to force a particular ordering.)
All I’m saying is that if we choose to implement some “weak edges” solution, we could take the chance and provide a more intuitive alternative to the <! operator.
Sorry, they are both an extra layer, which I think is workaround which should be unnecessary. If we’re going to rework this, let’s take the chance to get rid of that.
They absolutely did. They clearly expect it to happen in the order it was written, so the BufWr is before the BufRd. If that’s not the case, then how can we even talk about it being sorted ‘wrong’?
My point is, where there’s any possibility that it could go wrong, use the order it was written. It’s confusing to require the user to add additional code to say, “yes, I really meant it.” We should be able to ‘automatically deduce’ the intention from the code, or? Making them reiterate it seems like a failure to me.
We could try to detect graph segments that share no connections and force them to be evaluated in the “written” order. We would then need to apply the topological sort on each graph segment individually.
Personally I believe in “explicit is better than implicit”.
At the end of the day, I’m happy with any solution that makes the <! hack obsolete
If we start from the principle that we differ from user order only where it doesn’t matter we should be able to achieve that.
That may mean an implementation that uses slightly more wirebufs in some cases. I think that would be a price well worth paying, and I suspect very much in keeping with the original design intention. Ease of user comprehension, avoiding exceptions, and minimising surprise are all important considerations.
But SynthDef syntax is designed to be expressive and user-friendly, and that’s good, but this sometimes leads to implicit dependencies and execution order. The SynthDef DSL already requires some interpretation to turn each unit into a corresponding node in the graph. I understand this as part of the design.
The case with BufWr/BufRd seems like a typical situation in which some interpretation will occur. Most UGens have explicit dependencies defined by their connections. These are naturally strong edges in the graph and form the basis for topological sorting. Implicit dependencies (as some examples were mentioned here) should be used judiciously when implicit dependencies cannot be naturally resolved.
In terms of optimization, would be nice to have some numbers.
A wrinkle – now, I’m aware, basically nobody is seriously doing this – but we did have questions in the past about whether it’s always necessary to write calculations in order, and – it’s not strictly necessary.
(Note at the top of the example, freqs depends on detunes but detunes is written second )
So there’s an implicit design decision being made here, which rules out an expressive alternative.
A SynthDef explicit syntax would be something more like this. By the way, it would be a good idea to have a readable intermediate DSL, but unlike SynthDef DSL, ensuring explicit graph construction,
I use an experimental scheme-like and JSON-like version of this idea to exchange between languages and GUI
Microsoft Excel and Haskell are some of the few languages that let you structure your program in a way that allows functions and data types to be declared and used in arbitrary order. (the last example above shows that a little)
It’s a special kind of expressivity; Having a clean syntax with those features would be great.
To be clearer though, I guess we should say that the order is clear from the code rather than strictly written. This is also more trivially true in say Saw.ar(SinOsc.ar(1).range(440, 880)) in which the Saw is created but not written before the SinOsc.
I think in this case that principle remains equally true, despite the extensive obsfucation via all those interconnected functions resolving only at the end in a cascade of lazy evaluation. It is nevertheless possible to trace it through and determine the order indicated by the code.
Thinking about processes and sequences of actions may seem more intuitive. Still, if we look at natural languages, the most trivial thing is mixing left-side definitions (like where-clauses) and right-side ones. It’s only possible with this other way (“referential transparency,” as they say). That’s also the way we do algebra. Most computer languages are formally very odd compared to most other things, like how we speak and use algebra.
Of course, you have more freedom in writing your program, so one expects you to use this to clarify it, not obfuscate.
I don’t know how a synthdef like this would feel, but the little dsp code I did this way was not bad.
Related to this topic, Haskell uses graph representation to evaluate expressions and update nodes with their resulting value. All shared expressions are only evaluated once, which is more efficient.
So, GraphDef is halfway there, although the other half is not easy to get right.
How I’m thinking of handling this is making a UGenConnectionDelegate. A buffer UGenConnectionDelegate would record the last buffer access as the UGens are being added to the graph, and insert these weak edges to any newly added UGen. This works because the added order is a valid topological ordering.
In buffer read and write’s case this is going to be added weak edges when the new ugen is a different type (read or write) from the previous.
There’d be a similar system for bus access and message sending.
Each UGen has a (class)method that lists which UGenConnectionDelegate (or multiple) it needs. This would require an extra step for UGen authors.
// And example of a UGen that reads a buffer and sends message.
connectionDelegates { ^[BufferConnectionDelegate, MessageConnectionDelegate] }
bufferType { ^\read } // requires by the connection delegate
messageType { ^\send } // sim.
For backwards compatibility, we could have a panic mode for when none is specified and insert weak edges to everything.
This is a more granular and verbose that the current pure/non-pure, but it thinks it’s more flexible and explicit. And it hopefully makes that <! operator obsolete.
Side note, I never knew <! existed or was required, so I’ve been writing synthdefs wrong for years.
Also, integrating these weak edges and ’ type’ information into the topological sorting process is not a bad idea (I have a topologicalSortWithWeakEdges function here, that tries to take care of cases like those). By assessing these edges during sorting, we can maintain the current system’s efficiency (either operations are not that expensive) while introducing a bit more fine possibility. But having to hack the ugen code seems a bit demanding.
I’m wondering how that would work in practice. How can you know that a UGen writes to a Buffer? Yes, you know that BufWr does, but what about extensions?
Also keep in mind that Buffers are not the only possible implicit dependency between UGens. For example, a UGen plugin may implement its own shared resources.
AFAICT there are really only two alternatives that cover all possible cases:
try to maintain the (implicit) written order of operation
ask the user to explicitly mark implicit dependencies
Each ugen has to declare this. If it doesn’t, we can’t assume anything. So we call panic on all the connection delegates and connect it to everything being held. This ensures everything works, but creates more wirebufs if the UGen class hasn’t been updated.
They should make their own connection delegate for this resource and register it with synthdef in initClass.
To be honest, this all sounds rather complex… I think a simple .after method would cover all cases and it would be easy to implement. Yes, it puts the burden on the user, but IMO it’s not a difficult thing to learn. Or maybe I’m just used to it because I’m coming from Pd.
Let’s just be careful not to overengineer things. Always envision yourself reading the code in 10 years
Another important thing to consider: once we add things like UGenConnectionDelegate not only as an implementation detail, but as part of the public API, we will be stuck with it forever!
At this point we should be very careful about adding methods or interfaces to the Class Library. (Of course, this also applies to my hypothetical .after method.)
They clearly expect it to happen in the order it was written, so the BufWr is before the BufRd. If that’s not the case, then how can we even talk about it being sorted ‘wrong’?
)
that’s amazing!