Maximum value for numwireBufs

Questions
#1

Hello,

For a project with very big synthDefs, I have to set very high values for the option numWireBufs of the server. Otherwise, I get an error mentioning I have to increase this value.

And I would like to know which is the maximum value I can set for numWireBufs (on both OSX and Linux) ?
Because I have remarked that setting very high numWireBufs values is OK on OSX, but generates GREY count errors on Linux.

I do not have a linux computer right now, but which is the max value for numWireBufs on Linux I can have without generating GREY count errors ?

s.options.numWireBufs = (8192*12000); s.boot;

Many thanks,

Christophe

#2

And I would like to know which is the maximum value I can set for numWireBufs

I don’t think there’s a maximum value.

but generates GREY count errors on Linux.

What do you mean by that? Can you post an example?

#3

What is the action you’re taking, right before the grey count errors? Because there are a lot of moving parts in your scenario, so it’s hard to guess based on “I did something (unspecified) and got this message.”

hjh

#4

Grey count errors are a part of sclang (related to garbage collection), not the audio server, so the wirebuf count should be completely unrelated.

If your SynthDef is complex/large enough that you’re having to set an extremely high wirebuf count, it’s possible that you’re uncovering a memory-related bug on Linux - this could result in a grey count error. If you have one mega-SynthDef you might consider trying to break it up into multiple smaller SynthDefs you can chain together - this could side-step any potential problems resulting from the SynthDef being too large.

If you have code that reliably reproduces the problem, consider filling out a bug report on the issue (and post the code of course).

Also, fwiw: I often use very complex SynthDefs, to the point that I can only run one or two copies at a time before maxing out my CPU. My numWireBufs is still only set to 4092… orders of magnitude smaller than what you’ve suggested. If you’re needing a count as high as 8192*12000, it might be that you’ve got a bug in your SynthDef that’s generating a lot more complexity than you intend to. You might try using assertChannels in your SynthDef to ensure that you actually have the channel counts that you expect. (extAssertShape.sc · GitHub)

SynthDef(\channelTest, {
	var sig;
	sig = SinOsc.ar([100, 200, 300]);
	sig.assertChannels(3);  	// expect an array[3]
	sig = [sig, sig];
	sig.assertChannels(2, 3); 	// expect an array[2] of array[3]'s
	
	sig = SinOsc.ar([100, 200, 300, 400]);
	sig.assertChannels(3); 		// this errors because sig is 4 channels now
});
#5

Did you try a power of 2 also ?

#6

SynthDef’s topological sort is designed to minimize wirebuf usage (that is, it prefers narrow over wide graphs). This may increase usage of CPU cache over RAM, improving processing speed. It performs best on the very common case of parallel chains of UGens, mixed down at the end.

(
SynthDef(\test, {
	var sig = Saw.ar((100 .. 199));
	sig = LPF.ar(sig, (100 .. 199) * 5);
	Out.ar(0, sig.sum)
}).dumpUGens;
)

[ 0_Saw, audio, [ 100 ] ]
[ 1_LPF, audio, [ 0_Saw, 500 ] ]
[ 2_Saw, audio, [ 101 ] ]
[ 3_LPF, audio, [ 2_Saw, 505 ] ]
[ 4_Saw, audio, [ 102 ] ]
[ 5_LPF, audio, [ 4_Saw, 510 ] ]
[ 6_Saw, audio, [ 103 ] ]
[ 7_LPF, audio, [ 6_Saw, 515 ] ]

Each Saw → LPF can use 1 wirebuf, and they’re separate chains, so you need 4 wirebufs so far.

Note that the SynthDef code creates 100 Saws first, then 100 LPFs, so you’d think this is a wide graph, but the topo-sort links them up into vertical chains.

[ 8_Sum4, audio, [ 7_LPF, 1_LPF, 3_LPF, 5_LPF ] ]

Now these 4 are mixed down: back to 1 wirebuf. And you don’t need these LPFs for anything else, so this can reuse one of them.

[ 9_Saw, audio, [ 104 ] ]
[ 10_LPF, audio, [ 9_Saw, 520 ] ]
[ 11_Saw, audio, [ 105 ] ]
[ 12_LPF, audio, [ 11_Saw, 525 ] ]
[ 13_Saw, audio, [ 106 ] ]
[ 14_LPF, audio, [ 13_Saw, 530 ] ]
[ 15_Sum4, audio, [ 14_LPF, 8_Sum4, 10_LPF, 12_LPF ] ]

Mix in 3 more chains – but these can reuse wirebufs, so in theory we’re still using only 4.

… and so on, all the way down.

233 units, but could be handled with only 4 wirebufs.

If you need ten million wirebufs, you have a very interesting, odd case that the devs should check out (so the absence of code sharing in this thread limits the scope of the investigation).

hjh

2 Likes
#7

Buuuut… I just found a case where multichannel expansion → mixdown produces a wide graph: (a * b).sum can result in a chain of MulAdd units, and it puts all of the a units upfront (requiring one wire buffer per channel).

SynthDef('wide!', {
	var n = 100;
	var a = SinOsc.ar(440 * (1..n));
	var b = LFSaw.kr({ ExpRand(1, 4) } ! n).range(0, 1);
	Out.ar(0, (a * b).sum)
}).add;

exception in GraphDef_Load: exceeded number of interconnect buffers.

// check the structure
SynthDef('wide!', {
	var n = 10;
	var a = SinOsc.ar(440 * (1..n));
	var b = LFSaw.kr({ ExpRand(1, 4) } ! n).range(0, 1);
	Out.ar(0, (a * b).sum)
}).dumpUGens;

[ 0_SinOsc, audio, [ 440, 0.0 ] ]
[ 1_SinOsc, audio, [ 880, 0.0 ] ]
[ 2_SinOsc, audio, [ 1320, 0.0 ] ]
[ 3_SinOsc, audio, [ 1760, 0.0 ] ]
[ 4_SinOsc, audio, [ 2200, 0.0 ] ]
[ 5_SinOsc, audio, [ 2640, 0.0 ] ]
[ 6_SinOsc, audio, [ 3080, 0.0 ] ]
[ 7_SinOsc, audio, [ 3520, 0.0 ] ]
[ 8_SinOsc, audio, [ 3960, 0.0 ] ]
[ 9_SinOsc, audio, [ 4400, 0.0 ] ]  -- here we're up to 10 wirefbufs
[ 10_ExpRand, scalar, [ 1, 4 ] ]
[ 11_LFSaw, control, [ 10_ExpRand, 0.0 ] ]
[ 12_MulAdd, control, [ 11_LFSaw, 0.5, 0.5 ] ]
[ 13_ExpRand, scalar, [ 1, 4 ] ]
[ 14_LFSaw, control, [ 13_ExpRand, 0.0 ] ]
[ 15_MulAdd, control, [ 14_LFSaw, 0.5, 0.5 ] ]
[ 16_*, audio, [ 1_SinOsc, 15_MulAdd ] ]   -- second channel is complete
[ 17_MulAdd, audio, [ 0_SinOsc, 12_MulAdd, 16_* ] ]  -- 1+2 are complete
[ 18_ExpRand, scalar, [ 1, 4 ] ]
[ 19_LFSaw, control, [ 18_ExpRand, 0.0 ] ]
[ 20_MulAdd, control, [ 19_LFSaw, 0.5, 0.5 ] ]
[ 21_MulAdd, audio, [ 2_SinOsc, 20_MulAdd, 17_MulAdd ] ]  -- +3
...

So * with + is a special case (but a very common one!). I’d go so far as to call this a bug (or, at least, suboptimal behavior).

hjh

1 Like
#8

Hello,

Thanks to all for the answers and still sorry for my long delay for answering (I was lost with this and other things).

Unfortunately, I cannot post a simple example, since it is linked to a big experimental project for spatial performance, I have been working on for several years, that I will announce very soon on this forum.

@scztt The reason why I wrongly linked high values of numWireBufs with grey count errors is because I noticed setting high values on Linux (above 1024) would generate automatically grey count errors with my program, whereas on Mac I can rarely have this issue.
That is why I set a limit for Linux on this line of the project.

Many thanks for your help,

Christophe