This isn’t a quark yet, but more like getting feedback on some (fairly concrete) ideas. For more background & motivation you could read the discussion using the plain old BinaryOpXStream, which is what you get when your use the .x adverb on an operator on streams. That thing has some limitations:
1.) Only operators/methods are supported (duh) not arbitrary (binary) functions. Although you can of course define as many operators as you like in SC, even at runtime, e.g.
('<>': { "hi".postln }) <> () // <> not defined in Event by default
And as far stream/patterns go, you can ‘x’ adverb any method taking one argument really (i.e. treat it as a binary operator) or even a method that takes more than one argument, but for those you get the default values for the rest, as for blend below (blendFrac is 0.5 by default)
Pbinop(\blend, Pseries(0, 10), Pseq([10, 100]), 'x').iter.nextN(8)
// -> [ 5.0, 50.0, 10.0, 55.0, 15.0, 60.0, 20.0, 65.0 ]
But it gets a bit tedious to have to define operators (or even pseudo-methods in events) just so you can use that .x adverb streamified, even though it’s a fun hack, e.g.
(Pbinop(\bhack,
Pbind(*[
bhack: { |self, other| other + rrand(1, self.numbr) },
numbr: Pseries(1, 2),
]),
Pseq([10, 100]),
'x').iter.nextN(8, ()))
// (e.g.) -> [ 11, 101, 11, 103, 15, 102, 16, 107 ]
2.) Not easy to get “variable row length” support. Recall that you can make Pseq (unlike quite a few other patterns) to “continue” past nils, e.g.
Pseq([1, 2, nil, 3, 4]).iter.all // -> [ 1, 2 ]
// but...
Pseq([1, 2, nil, 3, 4]).iter.nextN(6)
// -> [ 1, 2, nil, 3, 4, nil ]
So you may want/like to use that trick (useful with patterns ending in “p”, e.g. Psetp and friends) but alas that won’t work with BinaryOpXStream:
(Pseries(10, 10) +.x Pseq([1, 2, nil, 3, 4])).iter.nextN(6)
// -> [ 11, 12, 21, 22, 31, 32 ]
3.) The fact that e.g. +.x is non-commutative operator even on arrays (and even more so) on streams can be confusing. (The outer loop is the left-hand side argument, by the way.) So maybe having (better) named arguments like a Pattern usually has is a usability improvement too.
Soo… I actually have 2.1 drafts for how to make a better mousetrap in this area.
Pforp
The first idea is a rather straightforward lifting of BinaryOpXStream to a Pattern class that just fixes the above shortcomings… but alas it turned out to have somewhat odd semantics in the more “advanced” use cases I came up with. Some usage examples first (there’s class code include in a collapsible box later).
By default Pforp works like BinaryOpXStream
Pforp(Pseries(10, 10), Pseq([1, 2, nil, 3, 4]), (_+_)).iter.nextN(6)
// -> [ 11, 12, 21, 22, 31, 32 ]
except you pass a function instead of an operator (as 3rd arg). This function receives an item from the first stream and needs to combine/merge it with an item from the 2nd (inner) stream. (This function is basically what the innermost code block in a pair of nested for-loops would be in charge of doing in a typical imperative programming piece of code, hence the Pforp name of the pattern.)
A first minor improvement (over BinaryOpXStream), which is “on by default” is that the input stream value is also passed as 3rd argument to the merge/loop function, so you can write for example
(Pforp(Pseries(10, 10), Pseq([1, 2, nil, 3, 4]),
{|...aa| aa.sum}).iter.nextN(6, 100))
// -> [ 111, 112, 121, 122, 131, 132 ]
The 4th arg to Pforp is where it gets more interesting; this is nilsToReset (the inner stream), and the default value is 1; however, if you change that to 2 it starts to “see” past the first nil, but every nil is still used to pull a (new) value for the outer stream/pattern…
// recall the 1st example
Pforp(Pseries(10, 10), Pseq([1, 2, nil, 3, 4]), (_+_)).iter.nextN(6)
// -> [ 11, 12, 21, 22, 31, 32 ]
// Now
Pforp(Pseries(10, 10), Pseq([1, 2, nil, 3, 4]), (_+_), 2).iter.nextN(6)
// -> [ 11, 12, 23, 24, 41, 42 ]
So the “middle” nil in the Pseq no longer rests the stream, so final two values in Pseq’s array are used for the creating the “2nd row” with the 2nd item pulled from the outer stream (producing [23, 24] vs. [21, 22] in the default nilsToReset=1 example.) These “rows” (sub-sequences) in the Pseq, separated by nils, can obviously be of different length).
The somewhat odd part however is that a finite Pseq acts as having a tail of infinite nils at the end, so the first two of those nils act to reset the inner stream… but the fact there are two taken into account also means that the outer stream gets advanced twice on the “array end” (now, i.e. with nilsToReset=2). So that’s why you have no 30-something value in the output. It’s actually possible to work around this limitation in Pforp by using an infinite stream for the inner sequence too, but still with explicit nils in it to make “row advances” in the outer pattern:
// So instead of
Pforp(Pseries(10, 10), Pseq([1, 2, nil, 3, 4]), (_+_), 2).iter.nextN(12)
// -> [ 11, 12, 23, 24, 41, 42, 53, 54, 71, 72, 83, 84 ]
// We can do
Pforp(Pseries(10, 10), Pseq([1, 2, nil, 3, 4, nil], inf), (_+_), 2).iter.nextN(12)
// -> [ 11, 12, 23, 24, 31, 32, 43, 44, 51, 52, 63, 64 ]
There are no unintended skips of the outer pattern now, and we’re alternating between (nil-delimited) sub-sequence of the inner pattern. This approach may be good enough for a “intermediate” use cases.
But let’s say, as “advanced” usage, we also want to able to skip “rows” (outer stream values) and not reset the inner pattern’s stream.
// this won't work as intended
Pforp(Pseries(10, 10), Pseq([1, 2, nil, nil, 3, 4, nil], inf), (_+_), 2).iter.nextN(12)
// -> [ 11, 12, 31, 32, 51, 52, 71, 72, 91, 92, 111, 112 ]
Now we’re “back to square one” as two (explicit, this time) nils in Pseq cause an inner stream reset (beside skipping a “row” from the outer stream). So, let’s make nilsToReset=3 and see what happens
Pforp(Pseries(10, 10), Pseq([1, 2, nil, nil, 3, 4, nil], inf), (_+_), 3).iter.nextN(12)
// -> [ 11, 12, 33, 34, 41, 42, 63, 64, 71, 72, 93, 94 ]
Managed to skip the 20s as intended, and the 30s “paired” with the right values now (3 and 4).
So Pforp is reasonably useable, but it can be a bit hairy to reason about, and having to use infinite Pseq sub-sequences to get the intended results in the “skippy use cases” is a bit non-intuitive as well.
`Pforp` implementation/code
Pforp : Pattern {
var <>outerPattern, <>innerPattern, <>mergeFunc, <>nilsToReset = 1;
*new { arg outerPattern, innerPattern, itemCombineFunc, nilsToReset = 1;
^super.newCopyArgs(outerPattern, innerPattern, itemCombineFunc, nilsToReset)
}
embedInStream { arg inval;
var outerStr = outerPattern.asStream, innerStr = innerPattern.asStream;
// could treat mergeFunc as a pattern too, but it's not clear if that helps much
// since functions can pull from streams "on their own" anyway...
var outerVal = outerStr.next(inval), innerVal;
var nilCount = 0, mustReset;
if (outerVal.isNil) { ^nil; };
loop {
innerVal = innerStr.next(inval);
if (innerVal.isNil) {
while {
nilCount = nilCount + 1;
mustReset = (nilCount >= nilsToReset);
// always advance outer on a nil
outerVal = outerStr.next(inval);
if (outerVal.isNil) { ^nil };
if (mustReset) { innerStr.reset };
innerVal = innerStr.next(inval);
innerVal.isNil && (mustReset.not) // repeat condition
};
// PforEach(1, nil, {}) would hang without next check
if (innerVal.isNil) { ^nil } { nilCount = 0 };
};
// Copies prevent the function from changing the args "too much".
// Well, to some extent, it's not a deep copy.
inval = yield(mergeFunc.value(outerVal.copy, innerVal.copy, inval.copy));
};
}
// TODO: handle cleanups (for event streams)
// todo: storeOn
}
As a minor improvement we can make Pforp “steal a nil” for the purposes of letting use a finite Pseq “most of the time”. (I’m changing the “running example” to something a bit simpler now). Basically, we’d like two write the 2nd line below, but get the results like for the first.
Pforp(Pseries(), Pseq([0, 10, nil, 100, nil], inf), (_+_), 2).iter.nextN(4)
// -> [ 0, 10, 101, 2 ]
// But:
Pforp(Pseries(), Pseq([0, 10, nil, 100]), (_+_), 2).iter.nextN(4)
// -> [ 0, 10, 101, 3 ]
There’s one-line hack in Pforp that will give us this. Insead of unconditionally pulling an item from the outer stream on every nil from the inner stream we do it only for nils that aren’t about to reset the inner Stream (this is how we “steal” one), but for the base case where a single nil would reset, we can’t obviously do that:
if(nilsToReset <= 1 || mustReset.not) { outerVal = outerStr.next(inval) };
For clarity I’m calling this modification Pforp2 below.
`Pforp2` implementation/code
Pforp2 : Pattern {
var <>outerPattern, <>innerPattern, <>mergeFunc, <>nilsToReset = 1;
*new { arg outerPattern, innerPattern, itemCombineFunc, nilsToReset = 1;
^super.newCopyArgs(outerPattern, innerPattern, itemCombineFunc, nilsToReset)
}
embedInStream { arg inval;
var outerStr = outerPattern.asStream, innerStr = innerPattern.asStream;
var outerVal = outerStr.next(inval), innerVal;
var nilCount = 0, mustReset;
if (outerVal.isNil) { ^nil; };
loop {
innerVal = innerStr.next(inval);
if (innerVal.isNil) {
while {
nilCount = nilCount + 1;
mustReset = (nilCount >= nilsToReset);
// cond to pull: not on the "reset-triggering nil" unless
// there's no other way to pull from outer
if(nilsToReset <= 1 || mustReset.not) { outerVal = outerStr.next(inval) };
if (outerVal.isNil) { ^nil };
if (mustReset) { innerStr.reset };
innerVal = innerStr.next(inval);
innerVal.isNil && (mustReset.not)
};
// PforEach(1, nil, {}) would hang without next check
if (innerVal.isNil) { ^nil } { nilCount = 0 };
};
inval = yield(mergeFunc.value(outerVal.copy, innerVal.copy, inval.copy));
};
}
// TODO: handle cleanups (for event streams)
// todo: storeOn
}
So what this buys us is
Pforp(Pseries(), Pseq([0, 10, nil, 100, nil], inf), (_+_), 2).iter.nextN(4)
// -> [ 0, 10, 101, 2 ]
// But:
Pforp(Pseries(), Pseq([0, 10, nil, 100]), (_+_), 2).iter.nextN(4)
// -> [ 0, 10, 101, 3 ]
Pforp2(Pseries(), Pseq([0, 10, nil, 100]), (_+_), 2).iter.nextN(4)
// -> [ 0, 10, 101, 2 ]
But alas Pforp is still wired in some “more advanced” cases when want to allow “skips in the middle”
Pforp2(Pseries(), Pseq([0, 10, nil, nil, 100]), (_+_), 3).iter.nextN(9)
// -> [ 0, 10, 102, 4, 14, 106, 8, 18, 110 ]
The 1 value output from the Pseries is skipped on purpose here in the jump from 10 to 102 (the explicit “double nil” in the Pseq causes that), but we’re also “missing” the 3 output from the Pseries because the end of the Pseq gives 3 nils, and we’re only “eating” one.
As it turned out, things can be made much simpler by distinguishing between the signal that advances the outer stream ane the signal that resets the inner stream. In fact, we can make everything simpler by using arrays for inner stream. Reaching the array end is now the signal to pull another (full array) from the inner steam and nils embedded in the array(s) still control the outer stream “pulls”. That gets us Pforai–see next post.
