Easy way to Pbind nil to a key (i.e. delete it) without terminating the stream?

Questions
1

It might not seem very useful to be able to delete keys from an event or event-stream, but if you filter event-streams, it is sometimes useful to unset a key. For example, if \freq is set “upstream” in an event-stream chain any setting of \degree etc. downstream will be ineffective when the Event is played (with the default Event.play machinery).

Obviously you can delete a key using a Prout or Pfunc. But requires an extra filtering step (or dropping Pbind[s] and “doing everything” in something else Prouts, Pxfade or a similar event-merging framework [with a suitable notion of merges, basically at least left- or inner joins supported, i.e. Dictionary.blend(fill: false) in SC’s “terminology”])

But, unless I’m mistaken, there doesn’t seem to be a way to delete a key from within a Pbind, because the first nil on any key ends the whole stream produced by Pbind, i.e.

Pseq([2, nil, 3]).asStream.nextN(3,()) // -> [ 2, nil, 3 ]

Pbind(\foo, Pseq([2, nil, 3]), \bar, 5).asStream.nextN(3, ()) 
// -> [ ( 'bar': 5, 'foo': 2 ), nil, nil ]

// not really deleting, but tolerates nils as key values:
(Prout({|ev| var fs = Pseq([2, nil, 3]).asStream;
	loop { ev = (foo: fs.next, bar: 5).next(ev).yield } }).asStream.nextN(3, ()))
// -> [ ( 'bar': 5, 'foo': 2 ), ( 'bar': 5 ), ( 'bar': 5, 'foo': 3 ) ]

// something that actually deletes from an "upstream"
(Pfunc {|ev| if(ev.foo == 0) {ev.foo = nil}; ev } <>
 Pbind(\foo, Pseq([2, 0, 3]), \bar, 5)).asStream.nextN(3, ())

Deleting a key from all events of a stream could be somewhat simpler with Pnaryop(\put, ... but Pnaryop also ends on nil for any arg:

p = Pbind(\foo, Pseq([2, 0, 3]), \bar, 5);
Pnaryop(\put, p, [\bar, 7]).asStream.nextN(3, ()) // ok, replaces each bar
Pnaryop(\put, p, [\bar, nil]).asStream.nextN(3, ()) // -> []

So has anyone run into this need, and what is/was your approach when you want to do this in the most concise way, i.e. delete a key but still mostly stick with Pbinds? E.g. has anyone (already) implemented a Pbind derivative that supports deleting keys in some concise syntax?

2

Quick thought: Pbind copies patternpairs at the beginning of embedInStream, before entering the loop. So, it looks like pairs will be stuck in there for the life of the embedded Pbind.

One way could be removing a pair from patternpairs, and adding a check to Pbind to make sure that key is still there before checking for nil.

// this check should go in Patterns.sc:344, or in a subclass of Pbind overriding embedInStream
var stillPresent = (patternpairs.indexOf(name) ? -1).even;
if (streamout.isNil and: stillPresent) { ^inevent };

Note: overriding embedInStream is practically equivalent to create a new subclass of Pattern

Then (and I guess this could be done in a wrapper Pattern class):

a = Pbind(\instrument,\default,\midinote,Pseq([nil]),\dur,1)
// remove midinote from patternpairs
a.patternpairs = a.patternpairs.asDict.reject{|p,n| n == \midinote}.asKeyValuePairs
3

This could be put in Pdel class (like Pkey, or even extend that to delete on an extra flag), but the most concise way I found so far involves moving the offending data key’s to a “junk”/dummy one:

~pdel = { |k| Pfunc { |ev| ev.removeAt(k) } }

(Pbind(\degree, Pseq((1..3)), \junk, ~pdel.(\freq)) <> (freq: 900, dur: 0.2)).play

(Pbind(\degree, Pseq((1..3)), \junk, ~pdel.(\freq)) <> (freq: 900)).asStream.nextN(4, ())

The following “optimization” which would avoid creating a junk key (and looks reasonably neat syntactically) doesn’t work alas because pdel deletes from event copy created by Pchain.

(Pbind(\degree, Pseq((1..3)) <> ~pdel.(\freq)) <> (freq: 900)).asStream.nextN(4, ())

You can actually avoid creating another key if you add any key in that Pbind e.g.:

(Pbind(\degree, ~pdel.(\freq), \degree, Pseq((1..3))) <> (freq: 900)).asStream.nextN(4, ())

This also gives [you] a hint why you’re deleting some key (because another specific one needs it not set).

Of note however, that this pdel approach done inside the Pbind can break if “the powers that be” decide that Pbind shouldn’t pass its internal event to its value sub-streams, but rather a copy like Pchain does. Then only the solutions external to Pbind would (still) work.

4

In 17 years of using SuperCollider, I don’t recall a case where I needed to remove something from an event that had been put there by something else.

The given scenario – “if \freq is set ‘upstream’ in an event-stream chain any setting of \degree etc. downstream will be ineffective when the Event is played” – is in theory a legitimate concern, but it hasn’t come up for me in practice because, for a specific pattern, I tend to choose one pitch representation and stick to it. That is, I’m writing either a diatonic pattern, or a chromatic pattern, or a pattern of frequencies. I won’t write do-re-mi as midinote 60, midinote 62, degree 2 because it’s unnecessarily confusing.

That said, it’s a valid data-processing operation. I’d suggest an alternative to nil. Nil is a concrete thing that represents the absence of a thing, and also represents the end of a stream. What if there were a different concrete thing that represents the absence of a thing but didn’t represent the end of the stream?

It’s probably too clever but you could almost use the Nil class for that: Pbind and similar patterns could say if(value === Nil) { event.removeAt(key) } { event.put(key, value) }. Or if there’s some problem with that, add a singleton class like Unset or Empty.

hjh

5

If one were to change/extend Pbind, another hack could be to allow assignments (binds) to nil. Presently

Pbind(nil, 199).asStream.next(()) // err

But conceivably you could make that bind/assignment to nil delete keys (even more than one) e.g. as

Pbind(nil, [\freq, \detunedFreq])

Another way would be to get rid of the following check and somewhat weird pseudo-error it creates (weird because it actually returns something…)

if (name.size > streamout.size) {
  ("the pattern is not providing enough values to assign to the key set:" + name).warn;
  ^inevent
};

And make it nil/delete the keys for which no values are provided instead. So you could then e.g. write

Pbind([\freq, \detunedFreq], [])

to delete keys. Although this looks less weird than binding to nil, it’s probably objectionable in that it removes an error checking mechanism (more useful in case your arrays are generated.)

6

Actually, that seems unlikely to happen because it would break the “side-channel” key-setting mechanism in Pn

e = ()
r = Pn(1, 3, \boo).iter

r.next(e) // -> 1
e // -> ( 'boo': true )

Basically, the Pn-Pgate-within-Pbind examples from the help page(s) rely on this ability of Pn(... \key) to change the Pbind’s internal (accumulated) event.

(
Pbind(
    \degree, Pn(Pseq((0..7)), inf, \step),
    \mtranspose, Pgate(Pwhite(0,5), inf, \step),
    \dur, 0.2
).play
)

So, there could be a Pdel or even Pkey itself (with a flag) with this ability to delete a key like that, relying on the same mechanism used by Pn. (Of some note, Pn doesn’t delete its key, but sets it to false when it no longer wants to assert it.)

By the way, how that help example works looks “obvious” if you trace its output

( 'degree': 0, 'dur': 0.2, 'mtranspose': 0, 'step': true )
( 'degree': 1, 'dur': 0.2, 'mtranspose': 0 )
//...
( 'degree': 7, 'dur': 0.2, 'mtranspose': 0 )
( 'degree': 0, 'dur': 0.2, 'mtranspose': 0, 'step': true )
( 'degree': 1, 'dur': 0.2, 'mtranspose': 0 )
//...

Although the \step: true appears to be “deleted” from the event, that really only happens because the EventStreamPlayer (ESP) re-copies the protoEvent on every event it “runs through” the stream.

Pn itself actually does just:

	embedInStream { | event |
		if(key.isNil) {
			repeats.value(event).do { event = pattern.embedInStream(event) };
		} {
			repeats.value(event).do {
				event[key] = true;
				event = pattern.embedInStream(event);
			};
			event[key] = false;
		};
		^event;
	}

So if we don’t “feed it” new copies of the input event, the true key remains asserted:

r = Pn(Pseq([1, 2]), inf, \boo).iter
e = ()
[r.next(e), e] // -> [ 1, ( 'boo': true ) ]
[r.next(e), e] // -> [ 2, ( 'boo': true ) ]

// But what ESP does is actually like
r = Pn(Pseq([1, 2]), inf, \boo).iter
e = (); [r.next(e), e] // -> [ 1, ( 'boo': true ) ]
e = (); [r.next(e), e] // -> [ 2, (  ) ]

Also, the result of that key set to false is not actually seen in the (inf) examples from the Pn help page. In fact with next calls that’s probably never the case, because the stream is “already done and outputting nils on next” by then (with the typical Routine implementation e.g. as done in Pattern.asStream).

r = Pn(1, 2, \boo).iter
e = ()
[r.next(e), e] // -> [ 1, ( 'boo': true ) ]
[r.next(e), e] // -> [ 1, ( 'boo': true ) ] (again)
[r.next(e), e] // -> [ nil, ( 'boo': false ) ]

So you’ll never really see that false in typical usage because most streams/patterns (and especially Pbind) end on the first nil they see. But if you Pseq (not Ppar!) two Pbinds then that false-setting is actually effective from one to the next because it works as in:

r = Pseq([Pn(Pseq([1, 2]), 2, \boo), Pfunc {|ev| ev.postln; 5} ]).iter
e = (); [r.next(e), e] // -> [ 1, ( 'boo': true ) ]
e = (); [r.next(e), e] // -> [ 2, (  ) ]
e = (); [r.next(e), e] // -> [ 1, ( 'boo': true ) ]
e = (); [r.next(e), e] // -> [ 2, (  ) ]
e = (); [r.next(e), e] // -> [ 5, ( 'boo': false ) ] ; posts ( 'boo': false )
e = (); [r.next(e), e] // -> [ 5, (  ) ] ; posts ( )
e = (); [r.next(e), e] // -> [ 5, (  ) ] ; posts ( )

But it’s not clear to me that there’s a practical application for this… other than for “empty” Pns whose sub-pattern does no yields so right after setting the key to true it needs to set it (back) to false, e.g.

r = Pn(_.().p, 1, \boo).iter
e = (); [r.next(e), e] // -> [ nil, ( 'boo': false ) ]
e = (); [r.next(e), e] // -> [ nil, (  ) ]

r = Pseq([Pn(_.copy.p, 2, \boo), Pfunc {|ev| ev.postln; 5} ]).iter
e = (); [r.next(e), e] 
// posts ( 'boo': false ), which is good, even though
// -> [ 5, ( 'boo': true ) ]
7

It would be wrong, in EventStreamPlayer, to reuse the same event object for subsequent evaluations. It must be a copy here. e = (); [r.next(e), e]; [r.next(e), e] isn’t exactly right usage.

I’m not following your reasoning here. If you copy the input event, and pass the copy to the child stream, and the child stream modifies the copy, and then Pbind yields the copy, then the copy and the yielded result would reflect the modification…? (To copy, modify the copy and yield the original would be not very smart, we certainly wouldn’t do that.)

hjh

8

There’s a somewhat subtle difference between what Pbind does and what Pchain does with the internal/intermediate event it uses (I’m not talking not the input event!) Pbind passes this intermeidate to value-patterns or the “right-hand side” of its assignments and then reuses it (no copies). Pchain doesn’t do that with its internal event. So, for example

Pbind(\iamhacked, Pfunc{|ev| ev[\morestuff] = 42; true}).iter.next(())
// -> ( 'morestuff': 42, 'iamhacked': true )

The key morestuff here is set because not only does Pbind read the value returned by the righ-hand-side of \iamhacked to get that key’s value, but also the internal event used by Pbind that was passed to the Pfunc in |ev| is not a copy, but Pbind’s “working event”. So you can also modify that with side-effects from the right-hand side.

If the 2nd to last line in Pbind’s snippet below…

			forBy (0, endval, 2) { arg i;
				var name = streampairs[i];
				var stream = streampairs[i+1];
				var streamout = stream.next(event); // no copy here
                // ... eventually (ignoring array assignments)
				event.put(name, streamout);
            }

… were changed to

				var streamout = stream.next(event.copy);

then one would no be able to use right-hand-sides with side effects on this intermediate event event in Pbind, i.e. morestuff would not be set in the Pbind’s result in the example above. But if someone were to make this change to Pbind, it would break Pn’s 3rd-arg key setting (too), because it relies exactly on there not being a copy done.

Now for contrast with Pchain etc. where such side-effects don’t work

Pchain(Pfunc{|ev| ev[\morestuff] = 42; (nay: false)}).iter.next(())
// -> ( 'nay': false )
Pchain(Pbind(), Pfunc{|ev| ev[\morestuff] = 42; (nay: false)}).iter.next(())
// -> ( 'nay': false )

Pseq is also somewhat hackable, like Pbind, when it switches between arrayed sub-streams:

Pseq([Pfuncn{|ev| ev[\morestuff] = 42; 66}, Pbind()]).iter.nextN(3, ())
// -> [ 66, ( 'morestuff': 42 ), ( 'morestuff': 42 ) ]

The input event, as modified by the arrayed-stream-element is passed onto the next element as its input (instead of a “pristine” copy of Pseq’s input event). That’s possible because

			repeats.value(inval).do({ arg j;
				list.size.do({ arg i;
					item = list.wrapAt(i + offsetValue);
					inval = item.embedInStream(inval);
				});
			});

Again if that last line were changed to just

					item.embedInStream(inval);

this side channel in Pseq (between its substreams) would disappear. (This one is slightly more subtle than what happens in Pbind because embedInStream in most patterns returns the input event, so if that’s changed internally in the stream/pattern, the change is reflected in the value returned by embedInStream to the embedder-caller (quite apart from what gets yielded). This mechanism affects patterns like Pseq that do “wholesale” sub-stream embedding by calling embedInStream instead of calling next iteratively.)

I’m not saying these changes should be done, but sometimes it helps to know where events are copied and where not, internally in the classlib. (And they’re not copied quite everywhere.)

9

That would definitely be wrong (IMO).

I even have doubts about that .copy in Pchain. My instinct is that the copy should be done at the top level (input event) and we should avoid copies at lower levels, unless it’s really bad not to copy. Agreed that it’s inconsistent currently; I think the way to resolve the inconsistency would be to remove copies wherever possible.

hjh

1 Like
10

I suppose the input-event zero-copy between sub-patterns semantics (tentative convention) can become unclear outside of ListPatterns and similarly “listy” patterns like Pchain or Pbind where the order execution of sub-patterns is easily predictable by the user.

In particular, I suspect Ppar could be nasty if did that… because it’s usually not going to be clear (to the user) which stream would pass input-event changes to the other… Ppar presently does “substream input-event isolation” like Pchain. Most of the questions reagarding “data-sharing between Pbinds” generally involve Ppar.

(r = Ptpar([
	0.0, Pfunc{|ev| ev[\cantseeme] = 42; (dur: 0.31)},
	0.1, Pbind(\dur, 0.2, \localonly, 42),
]).iter)

10 do: {r.next(()).postln}
// ( 'delta': 0.1, 'dur': 0.31 )
// ( 'delta': 0.2, 'dur': 0.2, 'localonly': 42 )
11

Ppar should never do so, because it returns only one of the sub-stream events at a time. There is no case where Ppar evaluates several of its sub-streams on one ‘next’ call (and if there is, it’s by definition a bug). So the isolation is in the event-prototype copy passed down from the EventStreamPlayer.

hjh

1 Like