Fixing Pproto's user-defined cleanup function behavior (and "double free" problem)

Here’s my stab at fixing

which is also a problem I’ve also rant into. Since it turns out that the user-supplied cleanup is altogether in all circumstances right now, I think it would okay to implement any sensible behavior (i.e. ignore the “specification” from the present documentation.)

To cut to the chase, my idea is to allow the user maximum flexibility to reuse the auto-generated cleanup function (and the “cleanup info bits” that its derived from) in their own cleanup. This basically means that the user-supplied function would more properly be called a functor as it would receive as input a (default-generated) cleanup function and return another (cleanup) function.

It was fairly simple to actually implement this as not much needs to happen in Pproto itself in this regard, but the resulting user code can be a bit hairy if it wants to re-process the “cleanup info bits”. Here are some usage example (I’ve implemented a new ProtoU class for now with my mod/idea, just so I can compare the behaviors in case of more dubious outcomes,)

p = Pbind(\degree, Pseq([2, 4], 3)); // we're not actually using the buffer here, but that doesn't matter 
(r = PprotoU(
	{ ~bufnum = (type: \sine1, amps: 1.0/[1,2,3,4,5,6] ).yield; }, p,
	{ |dfc, cla| cla.postln; { |flag| "Actual cleanup!".postln; dfc.value(flag) } }
).play)

So the basic idea: the user-supplied cleanup functor has “two stages” in this trivial example: it prints something which we’ll ignore for now (it’s actually the cleanup “info bits” list) and then return a function that calls the default cleanup function, not before it prints another message.

This code comes with a bit of a surprise, which is that it exposes another bug in the current implementation of Pproto, namely there’s “double free” type bug because bot Pproto and its inner Ptpar do ^cleanup.exit when the inner stream ends. So if you actually let the above code run to completion you’d see something like

[ ( 'bufNum': 1, 'numFrames': 1024, 'amps': [ 1.0, 0.5, 0.33333333333333, 0.25, 0.2, 0.16666666666667 ], 'numChannels': 1, 
  'server': localhost, 'type': buffer ) ]
Actual cleanup!
Actual cleanup!

But if you do r.stop before it’s done, then only one “Actual cleanup” is called/printed, because the EventStreamPlayer then does the correct business of calling the cleanup function only once (Pproto no longer gets to do any of that).

This issue aside (which I’m not yet sure where to fix, since Ptpar might be useful indepently), the “cleanup bits” can be used e.g. to do some custom actions on some cleanup subtypes. As my favorite example, here’s doing a custom cleanup for groups, i.e. release them first instead of the default insta-kill that Pproto does:

(r = PprotoU({ ~group = (type: \group).yield; }, p,
	{ |dfc, cla| cla.postln; { |flag|
		cla do: { |ev| switch (ev[\type],
			\group, { Group.basicNew(s, ev[\id]).release.postln }) };
		fork {3.wait; dfc.value(flag).postcln }}}).play;)

Here’s the link to the modified Pproto (PprotoU) that works with above examples:

A slightly more useful example perhaps: buffer “debugger”, i.e. plotter fired up on “stop” (or stream end) as a custom cleanup. The first/larges part of this chunk (i.e sythn and. p) just uses the parallel “self-modifying” buffer example from the Pproto help, with some minor tweaks.

SynthDef(\osc,{ arg out=0, bufnum=0, numbufs = 8, sustain = 1, freq = 500, amp = 0.1, pan = 0;
    var audio;
    audio = Osc.ar(bufnum, freq);
    audio = EnvGen.ar(Env.linen(0.01, 0.90,0.9), 1, timeScale: sustain, doneAction: Done.freeSelf) * audio;
    audio = Pan2.ar(audio, pan, amp);
    OffsetOut.ar(out, audio);
}).add;

(
p =  Ppar([ 
	Pbind(*[
		instrument:     \osc,
		freq:        Pwhite(1, 6) * 100,
		detune:        Pfunc { Array.fill(3.rand + 1, {3.0.rand}) },
		dur:        Prand([2,2,2.5,1],5), //10
		db:        Pn(Pstep([-10, -20, -20, -15, -20, -20, -20], 0.5) ),
		legato:        Pwhite(0.0,1).linexp(0,1,0.1, 3)
	]),
	Pbind(*[
		type:        \sine1,
		amps:        Pseg(Pfunc{ | ev | Array.fill(10, {1.0.rand}) }, 1),
		numOvertones:    Pseg(Pwhite(0, 9), 10).asInteger,
		amps:        Pfunc{ | ev | ev[\amps].copyRange(0, ev[\numOvertones]) },
		dur:         Pseq([1], 15), // 0.05
		bufNum:        Pkey(\bufnum)
	])
]);
)

(r = PprotoU(
	{ ~bufnum = (type: \sine1, amps: 1.0/[1,2,3,4,5,6] ).yield; }, p,
	{ |dfc, cla|  { |flag| cla do: { |ev| switch (ev[\type], \buffer, 
		{ { Buffer.new(s, ev[\numFrames], ev[\numChannels], ev[\bufNum])
			.plot.parent.onClose_({dfc.value(flag)});
	}.defer })};
}}).play)

r.stop // displays buffer(s) too
// need to keep buffer around until window closes or else 
// sclang crashes if you free a buffer it is displaying in a plotter
// beware of the "double free" bug in this context!

Of course this is a bit boring since there’s only on buffer used in this example…

I’m awaiting comments, especially on where the “double cleanup.exit” problem should be fixed, but also if you think the usage examples make it seem too hairy to have this level of forwarding to the user in terms of custom cleanups…

After looking at a few more implementations it’s clear to me now that Pfpar should call cleanup.exit and Pproto itself should not. I was a bit confused that Ppar doesn’t call cleanup.exit, but then Ppar doesn’t kill off any of its streams early, unlike Pfpar. On the other hand Pproto in itself has no logic to kill streams by itself.

This is how the “double free” happens: Pproto passes the protoEvent it just created (with all the auto-generated cleanups) down into Pfpar. So when one of Pfpar’s substreams actually ends, Pfpar sends up not only whatever its substream accumulated as cleanup but also what Pproto “sent down” into Pfpar (via the protoEvent). So then Pfpar does a cleanup.exit (which includes the Pproto generated cleanup event[s]) and then Pproto does exactly the same thing, i.e. a cleanup.exit on the same “cleanup stuff” it generated, kept a copy in its own cleanup object and sent down into Pfpar.

Figuring this out in the absence of a symbolic/stepping debugger was quite a headache…

To put it in far simpler terms, Pproto as viewed by who creates a stream instance of it, is (obviously) one stream. But because of its implementation (split in two classes) it keeps not one but two copies of an EventStreamCleanup instance, both populated with the self-generated cleanup actions. Hence the double free…