How to clean up?

During extensive SC sessions stuff piles up in the background, buffers hanging around in memory, nodes and groups you don’t want and stuff I don’t have a clue about.

Normally I do several .freeAlls like s.freeAllBuffers etc…

Even if I kill and reboot server, certain things seem still to be there. I’m not always sure what’s on the client side and what’s on the server side.

What are your methods to get a clean sheet without restarting server, SC or OS.

A few techniques from my personal practice…

Buffers

  • For buffers with data read from on-disk files, I have a class I use for caching them (e.g. a file from one disk location is only ever loaded once), so I never need to worry about freeing buffer unless I accidentally thousands of them / a few extremely large ones.
  • For cases where one buffer is needed per-synth (e.g. delay lines), use LocalBuf unless the buffers are really large.
  • When buffers are needed in a pattern context, allocate them in a Pproto so they’re cleaned up when the pattern is done.
  • On MacOS at least, leaving large unused buffers around in memory should have a negligible performance impact in almost all cases. Memory that hasn’t been read in a while is purged from active memory or compressed - both operations should have no performance impact unless you’re REALLY pushing things with tens of gb of data or extremely high CPU usage.

Groups

  • For Pattern use-cases, uses Pgroup, Pbus, or Pproto to manage groups, so they are freed when the pattern ends.
  • For long-running “global” groups (e.g. a chain like [synths] -> [effects] -> [output]), you can allocate in a ServerTree function. This ensures there is one and only one of each running at all times, and sets up your Group structure every time you do cmd+period.
 // run only once at startup
ServerTree.add({ ~groups.value }).run(\all);

// When you add groups to the function, either cmd+period to re-build all, or run the new line-of-code only
~groups = {
	~synthGroup = Group();
	~fxGroup = Group(~synthGroup, \addAfter);
}; 

Synths:

  • As much as possible, make sure SynthDef's have a \gate parameter that is hooked up to an envelope or some other UGen with a doneAction / Done.kr. This gives you a guarantee that Synths will end when \gate is set to 0. This gives you proper lifetime when using Event/Pattern workflows, and gives you something concrete to debug. If you have “leaked” Synths, use the ServerView quark or s.asGroup.dumpTree(true) to find synths with \gate, 1, and then determine why \gate, 0 was never sent to them.
  • For long-running Synths (e.g. a global EQ), use Ndef to manage Group and Bus details. You can also create these in a ServerTree function (above) or init Tdef (below).

Buses

  • Set a very high numBuffers. On normal desktop systems / normal use-cases, using a large number of buffers should have relatively little performance impact.
  • When buses are needed in a pattern context, use Pbus or Pproto to allocate them.
  • When long-running global buses are needed, allocate with a nil check: ~fxBus = ~fxBus ?? { Bus.audio(s, 2) }. You don’t need to alloc buses in ServerTree because buses are not freed by Cmd+Period.

General tips

  • The FreeAfter quark allows you to attach resources to Synths - they will be free’d when the Synth finishes:
~note = Synth(\foo, [\buf, ~buffer, \out, ~bus]);
~buffer.freeAfter(~note);
~bus.freeAfter(~note);
  • Break the habit of having an “initialization dance” where you run a bunch of scattered lines of code to initialize all the pieces of your setup. From experience, 90% of my clean-up issues come from having overly complicated initialization workflows - because I allocate things too often or fail to clean everything up.
  • As much as possible, maintain a SINGLE place where long-running resources are initialized. Make this initialization happen automatically, either on cmd+period by using ServerTree, or via a single function that you can run once to start a patch / setup, and free it.
  • If you want to create/free complex resources dynamically and at specific times, pair an init and free functions to do this. Tdef's can provide a global registry of init/free functions, and have a persistent environment attached for storing those resources. You can use this environment to keep track of resources you’ve allocated. An example:
Tdef(\initCoolSound, {
	|e| // <-- envir of the Tdef is passed in to the function here
	e.use {
		~buffer = ~buffer ?? { Buffer.alloc(s, 100000) };
		~group = ~group ?? { Group() };
	}
}).envir_(()); // <-- empty environment

Tdef(\freeCoolSound, {
	|e|
	e.values.do(_.free); // free all
	e.clear;             // clear my keys so they'll be initialized next time
}).envir_(Tdef(\initCoolSound).envir); // <-- share envir with my init Tdef


// Initialize
Tdef(\initCoolSound).fork; // run once
Tdef(\initCoolSound).envir.postln; // now we have resources
Tdef(\freeCoolSound).envir.postln; // same environment!
Tdef(\freeCoolSound).fork; // free everything
Tdef(\freeCoolSound).envir.postln; // now our envir is empty again

Tdef(\initCoolSound).get(\buffer); // access a resource

 // run these just once to automate setup/cleanup 
ServerTree.add(Tdef(\initCoolSound));
CmdPeriod.add({ Tdef(\freeCoolSound).fork });
4 Likes

Since 2005, I’ve been putting all of my sequencing into “musical process” objects: in the ddwChucklib quark, PR defines a “process prototype” and BP (“bound process”) binds the prototype to concrete data. Basically, PR is like a class and BP is like an instance, except they can be defined and redefined at runtime.

Then, the process is responsible for all of the persistent resources that it needs.

The process will play onto a bus? The process’s ~prep function creates the bus, and its ~freeCleanup function releases the bus.

It needs a buffer? ~prep loads it, ~freeCleanup frees it.

Once this is coded into the process object correctly, then I don’t have to think about it: PR(\myDrumProcess) => BP(\kick) loads buffers, allocates buses etc, and BP(\kick).free drops them. The musical processes take care of the details by themselves.

IMO the root of the “stuff piles up in the background, buffers hanging around in memory, nodes and groups you don’t want” problem is that patterns/Ndefs/Tdefs often rely on resources external to themselves – so, tracking the state of a pattern/Ndef/Tdef is not sufficient to manage resources. scztt is getting at this when he recommends Pproto, Pgroup, Pbus etc.

Once I solved that problem for myself, it stopped being an issue – for the last 15 years (that = confidence in the architecture).

hjh

1 Like

Personally, when i started SC, i had lot of problems with freeing synths, buffers and busses, because i used this kind of programming:

~mysynth = Synth();
~mybus = Bus.control(s,1);

Then when I change some code, I run it again, and leaked synth and bus piles up

Then I discovered patterns and Pdef: you can execute the code as many time as you want and there will still be just one instance

To imitate Pdef, i wrote a quark to give a name to every bus, buffer and group:

BusDef(\mybus, \control, 1);
BufDef(\sample1, "mysample.wav");
GroupDef(\mygroup);

I never free them because it is very rare for me to load so many of them that memory is full, since there is no more duplicate

2 Likes

That’s a very good idea!

hjh

1 Like

JITLibExtensions has BufEnvir.
http://quark.sccode.org/JITLibExtensions/help/BufEnvir.html
I find it kind of an odd interface and think the Pdef style a bit easier (I’ve also created my own BufDef style interface).

For buses and groups I just use Ndefs and never explicitly create either