Troubleshooting: numWireBufs & NamedControls problem

[ingo]

Reading another thread reminded me of a strange ‘situation’ I had in the past, so I dug up the code (below).

I had made two versions of the same code, one ‘normal’, the other using named controls. With the second one, \sk2, I can run less partials in the DynKlang Ugen before it errors on the number of interconnect buffers. In the startup file there is s.options.numWireBufs = 128;.

When I manually set numWireBufs again and reboot the server sk2 works better, that is the ~segments setting can be higher (than 21), but not as high as for \sk1 that can go to the max. Also the maximum setting varies.

Reducing the scene makes the problem go away. (Reduced too much?)

EDIT: Win11 & SC 3.12.2

// s.options.numWireBufs = 128; //set in startup file

(
~shuheiKawachi = {
	arg x, y, a, b;
	((cos(x) * cos(y))
	+ (cos((sqrt(a) * x - y) / b)
		* cos((x + (sqrt(a) * y) ) / b))
	+ (cos(( sqrt(a) * x + y) /b)
			* cos((x - (sqrt(a) * y*y)) / b)))/3;
};
)

(
~segments = 80;
Ndef(\sk1, {
	arg xMajor = 0.0
	, yMajor = 0.0
	, rMajor = 100
	, rMinor = 1.5
	, circlefreq = 0.0005
    , a = 2 * pi
    , b = 0.5
	;
	var arr, result, xMinor, yMinor, freqs, amps, phase;
	freqs = Array.exprand(~segments, 50, 1500).sort;
    phase = Array.fill(~segments, 0.0);
	xMinor = xMajor + (rMajor * SinOsc.ar(circlefreq));
	yMinor = yMajor + (rMajor * SinOsc.ar(circlefreq, pi / 2));
	amps = Array.fill(
		~segments,
		{
			arg seg;
			x = xMinor + (rMinor * sin(seg * 2 * pi / ~segments));
			y = yMinor + (rMinor * cos(seg * 2 * pi / ~segments));
			result = abs(~shuheiKawachi.(x, y, a, b)) / 12;
			result
		}
	);
	d = DynKlang.ar(`[freqs, amps, phase]);
	Out.ar(0, d!2)
});
)


(
~segments = 22; // #partials
Ndef(\sk2, {
	var rMajor = \rMajor.kr(100.0);
	var rMinor = \rMinor.kr(1.5);
	var circlefreq = \circlefreq.kr(0.0005);
	var result, xMinor, yMinor, freqs, amps, phase;
	freqs = Array.exprand(~segments, 50, 1500).sort;
    phase = Array.fill(~segments, 0.0);
	xMinor = \xMajor.kr(0.0) + (rMajor * SinOsc.ar(circlefreq));
	yMinor = \yMajor.kr(0.0) + (rMajor * SinOsc.ar(circlefreq, pi / 2));
	amps = Array.fill(
		~segments,
		{
			arg seg;
			x = xMinor + (rMinor * sin(seg * 2 * pi / ~segments));
			y = yMinor + (rMinor * cos(seg * 2 * pi / ~segments));
			result = abs(~shuheiKawachi.(x, y, \a.kr(2 * pi), \b.kr(0.5) )) / 12;
			result
		}
	);
	d = DynKlang.ar(`[freqs, amps, phase]);
	Out.ar(0, d!2)
});
)

Ndef(\sk1).clear;
Ndef(\sk2).clear;

// exception in GraphDef_Load: exceeded number of interconnect buffers.
// while reading file: '...\synthdefs\temp__0sk2-486857386_1049.scsyndef'exception in GraphDef_Load: exceeded number of interconnect buffers.
// while reading file: '...\synthdefs\temp__0sk2-486857386_1049.scsyndef'*** ERROR: SynthDef temp__0sk2-486857386_1049 not found
// FAILURE IN SERVER /s_new SynthDef not found

Reduced version,

(
~segments = 80;
Ndef(\sk1, {
    arg minamp = 0, maxamp = 1;
	var arr, result, xMinor, yMinor, freqs, amps, phase;
	freqs = Array.exprand(~segments, 50, 1500).sort;
    amps = Array.rand(~segments, 0, 1) * 0.05;
    phase = Array.fill(~segments, 0.0);
	d = DynKlang.ar(`[freqs, amps, phase]);
	Out.ar(0, d!2)
});
)

(
~segments = 80;
Ndef(\sk2, {
	var arr, result, xMinor, yMinor, freqs, amps, phase;
	freqs = Array.exprand(~segments, 50, 1500).sort;
    amps = Array.rand(~segments, \minamp.kr(0), \maxamp.kr(1)) * 0.05;
    phase = Array.fill(~segments, 0.0);
	d = DynKlang.ar(`[freqs, amps, phase]);
	Out.ar(0, d!2)
});
)

Ndef(\sk1).clear;
Ndef(\sk2).clear;

[ingo]

Just found something. Changed:

	xMinor = \xMajor.kr(0.0) + (rMajor * SinOsc.ar(circlefreq));
	yMinor = \yMajor.kr(0.0) + (rMajor * SinOsc.ar(circlefreq, pi/2));

to

	xMinor = \xMajor.kr(0.0) + (rMajor * SinOsc.kr(circlefreq));
	yMinor = \yMajor.kr(0.0) + (rMajor * SinOsc.kr(circlefreq, pi/2));

now \sk2\ works. But why? setting both to .ar does not work.

[jamshark70]

@ingo

I had made two versions of the same code, one ‘normal’, the other using named controls. With the second one, \sk2, I can run less partials in the DynKlang Ugen before it errors on the number of interconnect buffers.

I spent some time a few weeks ago on a related UGen-sorting problem.

I’m not sure I’ll have time to try to untangle your specific SynthDef (it’s a lot of math UGens), but some general notes.

The UGen sort works by “resolving” UGens into the final order. UGens are “available” to be resolved when either they have no inputs, or all of their inputs have already been resolved – that is, every time a UGen gets resolved into sort order, it checks that unit’s descendants, and any of those downstream units will be made “available” if they have no further unresolved inputs. In general it will try to go depth first, but there is at least one known case where the first part ends up being width first (using more interconnect buffers), and there is no obvious way to determine which is the optimal path to take.

My guess in your case is: When you use SynthDef function arguments, there’s one big Control UGen with all 7 arguments. This in general will be resolved first – in one go, this resolves many downstream inputs, potentially making a lot of UGens available to be added. With NamedControls, there is one UGen per synth input (7 in all, as separate UGens). After adding one or two of them, it will start working on any downstream units that have become available, and it will do as much of this as it can before moving on to the next NamedControl. When these UGens are added in a different order, the arrangement of wire buffers is likely to be different. In this case, it’s bad luck for the NamedControl version. Another case could easily have the opposite result.

I had tested an inverted topological sort, and one thing I found from this was: The original topo-sort handles some cases better, and my inverted sort handles other cases better, but neither version handled every case better. Moreover, in the less-efficient cases, I couldn’t see a fast-to-execute way to determine in advance which approach would use wire buffers more efficiently. So unfortunately, some of it is luck (and, there is no way to make the NamedControl version sort identically to the argument version). Some ways of writing a SynthDef will use more wire buffers for the same operations, and it’s exceedingly difficult to predict when that will happen.

My best recommendation at this time is to increase the number of wire buffers until it works.

hjh

PS Only ar units use wire buffers, so this is why kr units didn’t trigger the error.

[ingo]

@jamshark70

I’m not sure I’ll have time to try to untangle your specific SynthDef (it’s a lot of math UGens), but some general notes.

No need to untangle as your answer explains a lot.

It confirms what I understood more or less from other threads. NamedControls is not (just) syntactic sugar. It’s a different way.

The sorting I kind of understand, the language is parsed. A structure has to be build before/while rendering happens. How this structure is build varies.

@jamshark70

PS Only ar units use wire buffers, so this is why kr units didn’t trigger the error.

Ah.

Thanks!

[jamshark70]

For fun… here’s a small scale, concrete demonstration of the weird kinds of things that can happen with the topological UGen sort.

Here’s a good case:

(
d = SynthDef(\test, {
	var n = 3;
	var ctls = Array.fill(n, { |i|
		[
			("freq" ++ i).asSymbol.kr(110 * (i+1)),
			("amp" ++ i).asSymbol.kr(1 / (i+1))
		]
	}).flop;  // 'flop' puts all freqs in one subarray, amps in the other
	var sig = SinOsc.ar(ctls[0]) * ctls[1];
	Out.ar(0, sig.sum)
}).dumpUGens;
)

And I’ll annotate the dumpUGens output with two additional columns:

• For each audio rate unit, how many signals are added and how many are consumed (I’m assuming 1/ every ar UGen requires a wirebuf for its output, so +1 inherently, and 2/ if an ar unit takes an earlier ar unit as an input, and the earlier unit is not used anywhere else, then its wirebuf is released = -1).
• Total wirebufs in use at that point.

+/- wirebuf	wirebufs
		[ 0_Control, control, nil ]
+1	1	[ 1_SinOsc, audio, [ 0_Control[0], 0.0 ] ]
		[ 2_Control, control, nil ]
		[ 3_Control, control, nil ]
+1	2	[ 4_SinOsc, audio, [ 3_Control[0], 0.0 ] ]
		[ 5_Control, control, nil ]
+1 -1	2	[ 6_*, audio, [ 4_SinOsc, 5_Control[0] ] ]
+1 -2	1	[ 7_MulAdd, audio, [ 1_SinOsc, 2_Control[0], 6_* ] ]
		[ 8_Control, control, nil ]
+1	2	[ 9_SinOsc, audio, [ 8_Control[0], 0.0 ] ]
		[ 10_Control, control, nil ]
+1 -2	1	[ 11_MulAdd, audio, [ 9_SinOsc, 10_Control[0], 7_MulAdd ] ]
-1	0	[ 12_Out, audio, [ 0, 11_MulAdd ] ]

The above ordering is ideal: at 7_MulAdd, it collapses two oscillator-multiply chains down into one signal before introducing the third. If n = 10, or 50, or 200, the graph will not get wider because it will always follow this pattern: new chain → collapse, new chain → collapse.

But that way of defining the NamedControls looks clunky. Let’s separate the freqs and amps.

(
d = SynthDef(\test, {
	var n = 3;
	var freqs = Array.fill(n, { |i|
		("freq" ++ i).asSymbol.kr(110 * (i+1))
	});
	var amps = Array.fill(n, { |i|
		("amp" ++ i).asSymbol.kr(1 / (i+1))
	});
	var sines = SinOsc.ar(freqs);
	var sig = sines * amps;
	Out.ar(0, sig.sum)
}).dumpUGens;
)

+/- wirebuf	wirebufs
		[ 0_Control, control, nil ]
+1	1	[ 1_SinOsc, audio, [ 0_Control[0], 0.0 ] ]
		[ 2_Control, control, nil ]
+1	2	[ 3_SinOsc, audio, [ 2_Control[0], 0.0 ] ]
		[ 4_Control, control, nil ]
+1	3	[ 5_SinOsc, audio, [ 4_Control[0], 0.0 ] ]
		[ 6_Control, control, nil ]
		[ 7_Control, control, nil ]
+1 -1	3	[ 8_*, audio, [ 3_SinOsc, 7_Control[0] ] ]
+1 -2	2	[ 9_MulAdd, audio, [ 1_SinOsc, 6_Control[0], 8_* ] ]
		[ 10_Control, control, nil ]
+1 -2	1	[ 11_MulAdd, audio, [ 5_SinOsc, 10_Control[0], 9_MulAdd ] ]
-1	0	[ 12_Out, audio, [ 0, 11_MulAdd ] ]

… and all 3 SinOscs have clumped up to the top. So, right at the start, you will need at least n wirebufs, where the other SynthDef could increase n arbitrarily. (Edit: Arrayed NamedControls produce the same structure.)

So a small change in the manner of defining the inputs produced a radically different (and suboptimal) structure.

FWIW, my alternate sort handles both of the above SynthDefs identically – but, if you Splay.ar (that is, Pan2 each change into a stereo pair before summing), then the original sort works slightly better than the alternate! … Which I take to mean that, no matter what we do, there will always be some case that is not handled perfectly. What’s unfortunate about that is that it’s not at all easy to understand why the UGen sort is behaving as it does, thus, hard to predict the best way to write your code … just have to try it, and adjust when needed.

hjh