Increasing the limits? - classvars, Function args and ServerOptions

Since I understand that there are API changes coming for SC 3.15.0, I wondered if now might be a good time to look at increasing some of the fixed limits within SC.
(Sorry if these have already been discussed somewhere - I am not a developer.)

Classvars
When updating TX Modular (which is a large project with many classes), I found that I had used too many classvars and had to refactor all my classes by using dictionaries to hold multiple class variables within one object (thanks HJH for suggestion). This worked but took a fair bit of time.
I wondered if this limit was fixed or if it could be increased?

Function and Synthdef args/vars/classes
There seems to be a 256 limit to the number of args and vars within a Function. This is also true for the number of classes used.
You can reduce the number of vars by using 1 dictionary to hold many vars. But still sometimes more than 256 args are needed.
This makes it hard to make large synthdefs with lots of controls. I have sometimes had to use multiple Synthdefs to do what I want.
Could this limit be increased (e.g to 256 x 16)?

ServerOptions
I wonder if the defaults should be increased, specifically for:

  • numWireBufs - I quite often bump into the 64 default limit.
  • memSize - 8192 seems low for current hardware.

Are there any other (visible/hidden) limits in SC which could also be considered?

Best,
Paul

Thanks for the suggestions:

re serveroptions, we’ve increased this to uncontroversial values last summer:

    var <>memSize = 256000;
	var <>numWireBufs = 512;

though in my own startup.scd I go crazy and have memsize = 2.pow(20) (1Go), numWireBufs = 8192. No idea though if that’s sensible or not :).

The other things sound like something @jordan could look into or maybe already has.

You could use named controls instead

\ctl.kr

Changing the things you’ve mentioned requires adding or altering the bytecodes. I’m currently not working on the interpreter, I’m reworking the compiler so we can have good error messages finally. I don’t think this would be high on the priority list though because there are ways around it.

Re: limits on class vars and function args+vars – one reason why this hasn’t been changed is because: if your class or function is large enough that you need 300 variables, then it almost certainly should be broken down into smaller pieces.

There’s nothing significant about a 256-var limit, except that the index fits within 8 bits. The issue is really risk vs benefit. There’s always risk in changing the interpreter’s internals. It may be worth it if there’s enough benefit to users.

The benefit would be that users have more freedom to structure their code as they see fit. But the design strategies this would open up are all of a sort that computer science has rejected for decades, and which are universally discouraged.

If you need 300 variables in your class, chances are that it’s a god object: ā€œA God Object is a class that knows too much or does too much. It is characterized by having an excessive number of methods, attributes, or dependenciesā€¦ā€ (btw this page looks like AI text, but at least this bit is ok). Hitting a limit is an indication that some redesign, breaking the job down into smaller components, would be a good idea.

Similarly, some professionals say that, ideally, a function or method should fit on one screen (and some are more strict than that, suggesting at most 10-15 lines per function/method). Otherwise, it’s too long and complex. I certainly don’t follow this rule to the letter, but if it’s passing 100, 150 lines, it’s in the back of my mind that ā€œthis function’s a bit out of control.ā€ If the variable declarations alone fill an editor panel, it’s time to modularize and encapsulate.

I’m sympathetic to the case that this perhaps shouldn’t be SC’s decision to make. At the same time, enabling the construction of unwieldy, difficult-to-maintain monster functions also strikes me as an illusory benefit.

hjh

Definitely agree here regarding normal functions and classes… but do you think this still applies to synthdef controls?

Often, I will have one control with many related controls to change how it behaves in time. Something like…

\a.kr.lag(\alag.kr)

… but many more controls for different morphing strategies.

Creating these sub controls is often done with a function.

This easily explodes.

This is one of the (many) things I like about your Syn and Plug classes, as it lets you have nested namespaces for controls!

Ahhhh… I had forgotten about SynthDesc msgFuncs.

There is actually no limit on the number of controls in the server:

// barfs with a SynthDesc error,
// but it does actually write the file
(
d = SynthDef(\lottaControls, {
	var ctlNames = Array.fill(5000, { |i| ("ctl" ++ i).asSymbol });
	var ctls = Control.names(ctlNames).kr(Array.fill(5000, 0));
	Out.kr(1000, ctls.sum)
}).writeDefFile;
)

s.boot;

s.sendMsg("/d_load", SynthDef.synthDefDir +/+ "lottaControls.scsyndef");

x = Synth(\lottaControls);

// press ctrl-shift-t
NODE TREE Group 0
   1 group
      1001 lottaControls
        ctl0: 0 ctl1: 0 ctl2: 0 ctl3: 0 ctl4: 0 ctl5 ... ctl4996: 0 ctl4997: 0 ctl4998: 0 ctl4999: 0

x.free;

msgFunc is useful for Events, but not really for any other reason.

So we should probably change that hard error into a warning. SynthDesc and its msgFunc are not strictly required for baseline SynthDef usage.

Edit: I can also see in my head a way to de-optimize msgFunc for large numbers of controls. It would be slower but probably better than just choking with an error. Point being that there’s no reason why a limit on function arguments should limit the number of controls in a SynthDef (and I’m thinking it would be better for the code to make the effort to handle it, rather than just erroring out).

(TBH I don’t know what I’d do with a 300-control SynthDef, though.)

hjh

1 Like

Well, I had a crack at it tonight, but didn’t get all the way there, and now the IDE is in a state where it’s refusing to reindent a method after I added a level of braces.

	makeOneMsgFunc { |controlNames, suffix, index = 0, limit = 250|
		var names = 0;
		var string;
		if(index < controlNames.size) {
			string = String.streamContents {|stream|
			var controlName, name, name2;

Tab should obviously push the last line here forward by one tab, but it’s not doing anything. Everything compiles, I don’t have broken braces anywhere, just IDE failed.

Not gonna deal with unrelated bugs right now. Will pick this up again maybe tomorrow, maybe Sunday.

hjh

Thanks for info and advice.
Good to know about using named controls instead of args.
I’m definitely guilty of god objects - in fact I have a whole pantheon of them!
Sometimes things start small, but keep expanding - I don’t always remember to refactor sooner rather than later.
Best,
Paul

1 Like

BTW this is just about done, will PR it later today.

EDIT: Remove 255-arg limit for SynthDefs by jamshark70 Ā· Pull Request #7483 Ā· supercollider/supercollider Ā· GitHub

hjh

1 Like