DynGen - Dynamic UGen

DynGen is a meta-UGen which allows to write DSP code on the fly using EEL2 as JIT-compiled language. This is essentially the SC equivalent of the gen~ object from Max MSP.

An example on how to use it for halfing the amplitude of a given signal.

// start the server
s.boot;

// registers a dyngen script on the server with identifier \simple
~simple = DynGenDef(\simple, "out0 = in0 * 0.5;").add;

// spawn a synth which evaluates our script
(
Ndef(\x, {DynGen.ar(
	1, // numOutputs
	~simple, // script to use - can also be DynGenDef(\simple) or \simple
	SinOsc.ar(200.0)), // ... the inputs to the script
}).scope;
)

This example isn’t too interesting, but we can also use DynGen for oversampling and single sample feedback really easily - e.g. here is the code and a sound snippet for two cross-phase-modulated SinOscs, 128 times oversampled

(
~complex = DynGenDef(\complex, "
twopi = 2*$pi;

phaseA += 0;
phaseB += 0;

freqA = in0;
freqB = in1;
modIndexA = in2;
modIndexB = in3;

oversample = 128;

osSrate = srate * oversample;
incA = freqA / osSrate;
incB = freqB / osSrate;

sumA = 0;
sumB = 0;

// calculate subsaples
loop(oversample,
    phaseA += incA;
    phaseB += incB;
    // wrap phases between [0, 1)
    phaseA -= floor(phaseA);
    phaseB -= floor(phaseB);

    // apply cross-phase modulation
    phaseA = phaseA + modIndexA * sin(twopi * phaseB);
    phaseB = phaseB + modIndexB * sin(twopi * phaseA);

    // accumulate (for downsampling)
    sumA += sin(twopi * phaseA);
    sumB += sin(twopi * phaseB);
);

// scale down b/c of os
out0 = sumA / oversample;
out1 = sumB / oversample;
").add;
)

(
Ndef(\y, {
	var sig = DynGen.ar(2, ~complex, 
		\freqA.ar(200.0),
		\freqB.ar(pi*100),
		\modA.ar(0.02, spec: [-0.1, 0.1]) * 0.05 * Env.perc(releaseTime: \releaseTime.kr(0.2)).ar(gate: Impulse.ar(\offsetKick.kr(4.0))),
		\modB.ar(0.0, spec: [-0.1, 0.1]) * 0.05,
	);
	sig * 0.1;
}).play.gui;
)

It is also easy to write modulatable delay lines

(
~delayLine = DynGenDef(\delayLine, "
buf[in1] = in0;
out0 = buf[in2];
").add;
)

(
Ndef(\z, {
	var bufSize = SinOsc.ar(4.2).range(1000, 2000);
	var writePos = LFSaw.ar(2.0, 0.02).range(1, bufSize);
	var readPos = LFSaw.ar(pi, 0.0).range(1, bufSize);
	var sig = DynGen.ar(1, ~delayLine,
		SinOsc.ar(100.0),
		writePos.floor,
		readPos.floor,
	);
	sig.dup * 0.1;
}).play;
)

When the script of a DynGenDef is updated, it will automatically replace the running UGen, such that livecoding of DSP is now possible.

The UGen is still considered beta, so the interface is not stable and is open to change.
I still have some open design questions, maybe the community can help me out here.
I haven’t tried if it works on Windows, so would be great to get some feedback for it.

Looking forward to hear some new sounds and see some interesting pseudo UGens popping up :slight_smile:

Massive thanks to @Spacechild1 for helping me how to handle NRT/RT synchronization in a good way and giving the initial idea for this during the symposium.

37 Likes

I’ve hit the :heart: button but that represents about 0.0001% of my actual reaction to this post.

:exploding_head:

hjh

3 Likes

Yeah!!! (and some more characters)

Great work!

I just spammed your repo with a few issues :smiley: I think the API is great, but I have a few suggestions. In particular, I don’t think that the audio inputs should be provided as varargs, see DynGen UGen inputs should not be varargs · Issue #21 · capital-G/DynGen · GitHub.

1 Like

Actually, I’m not too happy about the add method, see concerns about the `add` method · Issue #22 · capital-G/DynGen · GitHub

In short, I would prefer something like:

// returns the DynGenDef bound to symbol \foo, creating it on demand
DynGenDef(\foo);

// get \foo def and send code to the Server
DynGenDef(\foo).load("some code");

// get \foo def and tell the Server to read the given file
DynGenDef(\foo).readFile("myCode.txt");

I don’t care too much about the actual method names, but more about the general pattern.

haha, i also wanted to double click the :heart:

That’s great news! Many thanks.

Thank you very much for this outstanding achievement.

I have just one question regarding the form in which this package will be delivered:

  • Will it be released in its current form?

  • Will it be included in the official build? (This seems unlikely due to licensing constraints.)

  • Will it be included in the official build? (it is unlikely possible due to the lisence…?)

  • Will it be incorporated into the sc3-plugins?

  • Or will it be distributed via quarks?

If it is to be incorporated into the official build, it would be wonderful to see SuperCollider expanding its support for external languages. For instance:

  • KaTeX integration within schelp — which has already been implemented

  • EEL2 support in sclang via quotation syntax — currently being developed as part of this contribution

Additionally, although still speculative, HTML rendering within schelp could be a truly valuable enhancement if considered in the future.

Once again, thank you for your remarkable work. I look forward to seeing how this evolves.

It may be premature to decide how a finished version would be released, since the interface hasn’t stabilized yet. “Official build” is a very premature question at this point (I think).

Maybe, but that might be a separate topic. I think that implementing an analog to MSP’s gen~ doesn’t directly have any bearing on importing other languages into other SC contexts.

EEL2 support in sclang – Jordan has done some work on language backend extensions, which would make it possible, but I wonder what features it would provide that SC doesn’t already have. Also, code-in-quoted-strings works if the code is likely not to have quotes in it (as seems to be the case for the type of DSP code in the top post here). My live coding dialect uses a lot of quote marks and is quite painful to write into SC string literals (which I don’t do in a live coding performance, but I do sometimes have to do if I’m running an automated timeline-sequencer that invokes live-coding strings). String literals are more likely to appear in EEL2 scripts doing sclang-type activities, so we’d want to consider carefully whether quoted strings are the right syntactic mechanism.

Or maybe an sclang extension here wouldn’t be a high priority (whereas a “live code sample-by-sample DSP” UGen does answer a specific need that’s been raised for years in the forum and prior mailing lists).

hjh

1 Like

Thanks for the feedback so far, much appreciated!

No - I think the current path of SC is to make things more modular to reduce maintenance work within the core, but there are some plans make working with such modules easier.
There are also no licensing issues. Using BSD licensed code in a GPL context is totally fine - the other way around would not be allowed.

No plans on doing that - sc3 plugins is already massive and I don’t want to put additional stuff in it - modularity is the way to go.

Since it needs to contain a binary it can’t/shouldn’t be distributed as a quark - but there are plans to make the installation and management of modules - such as quarks, server extensions and the 3.15 feature of sclang-extensions aka gluons - much more easy. Hopefully this will be part of the next 3.15 release, so stay tuned for further information on that.

This extensions does not have any kind of EEL2 support for sclang. You pass a string with EEL2 code from the language to the server, but in this way sclang has support for any language. As hjh pointed out, I don’t think that EEL2 support for sclang would be that interesting since it wouldn’t you enable to do new things, but there are plans on embedding tidal cycles into sclang → Embedding Tidal cycles in sclang

EEL2 syntax allows for strings, but currently it is not enabled within DynGen since I don’t know what it would be useful since the server has no way of handling strings.

A bit OT, but I think it is better to not tie ourselves too much on HTML for documentation since a web browser is a big dependency which I’d ideally like to remove at some point so we can have a smaller footprint of SC installs.

3 Likes

This particular point reminds me of a past discussion:

This is great, now I can play with my own reverb, without having to recompile C++. I will test Windows and Linux versions this week. Is there any limitation? Something that a compiled UGen can do that EEL can’t?

YES!!! So excited to use this. Just wondering if it’s possible to use use DynGen with sc buffers?

I tested on Windows 10, all the examples worked.

BTW: the first example on github has a syntax error, the coma after SinOsc is too much.

1 Like

Wow, it’s so good it put you in a coma? I’ll be extremely careful then

(apologies)

v0.2.0

Download via Release v0.2.0 · capital-G/DynGen · GitHub

Changelog

  • Added read and write methods for SuperCollider buffers (bufRead, bufReadL, bufReadC, bufWrite)
  • Allow loading code from dedicated file via DynGenDef.load
  • Print EEL2 compile errors
  • Changed DynGenDef interface to use .send instead of .add
  • Enforce audio rate on input parameters
  • rename scriptBuffer argument to script on DynGen.ar
  • Delete temp file for sending code to the server after usage

Since DynGen internally works with doubles instead of SuperCollider internal floats it is possible to go beyond the 32 bit address restrictions of BufRd by working with an internal DynGen phase which uses 64 bit floats.
This allows to address 2**53 samples accurately, which translates to around 6000 years on 48 kHz instead of the 6 minutes of BufRd

Interface is still non-final and will be tweaked for an upcoming 0.3.0 - primarily

See https://github.com/capital-G/DynGen/issues for all open discussions.


Good news, now it is.

Well, a compiled UGen will always have superior performance than a JIT compiled one - though I’d say that the performance of EEL2 is pretty good and will get you pretty far.
It is also possible to expose additional internal methods within the DynGen like the now implemented bufRead.

A dedicated C++ UGen also allows you to tap into APIs though, such as system APIs or e.g. network access.

I think it would be interesting which additional methods should be available within DynGen land - I’ll have to work more with the gen~ object to get to know which methods are handy.

9 Likes

I think the most useful gen~ objects are: phasor, noise, cycle, accum, clip, wrap, fold, min, max, abs, latch, delta, mix, switch, gate, selector and peek and poke with variable Interpolation.

1 Like

i had a go at translating a few of the gen inbuilts of them when I was working on some new stuff in eel, probably some bugs but maybe helpful starting point.

    function delta(in) local(init, prev)
    (
        !init ? (prev = in; init = 1);
        d = x - prev;
        prev = x;
        d;
    );

    function change(in) local(init, prev)
    (
        !init ? (prev = in; init = 1);
        d = in - prev;
        prev = in;
        sign(d);
    );

    function latch(in, pass) local(init, held)
    (
        !init ? (held = in; init = 1);
        pass ? held = in;
        held;
    );

    function sah(in, trig, thresh) local(prev, output, init)
    (
        !init ? (prev = 0; output = 0; init = 1);
        (prev <= thresh && trig > thresh) ? (output = in);
        prev = trig;
        output;
    );

    function scale(in, inlow, inhigh, outlow, outhigh, power)
    (
        inscale = (inhigh - inlow) != 0 ? (1.0 / (inhigh - inlow)) : 0;
        outdiff = outhigh - outlow;
        value = (in - inlow) * inscale;
        
        value = (value > 0.0) ? pow(value, power) : (value < 0.0) ? -pow(-value, power) : value;
        value = (value * outdiff) + outlow;

        value;
    );

    function linear_interp(x, y, a) ( x + a * (y-x));

    //alias
    function mix(x, y, a) (linear_interp(x, y, a));

    function mstosamps(ms)( srate * ms * 0.001 );

    function sampstoms(s)( 1000 * s / srate );

    function fixnan(x) ((x && x) ? x : 0);

    function DCBlock(in1) local(x1 y1)
    (
        y = in1 - x1 + y1 * 0.9997;
        x1 = in1;
        y1 = y;
        y;
    );

    function noise() (scale(rand(), 0, 1, -1, 1, 1));

    function wrap(x, low, high)local (range, result)
    (
        range = high - low;
        result = range != 0 ? (x - floor((x - low) / range) * range) : low;

        result;
    );

    function clip(x, low, high) local(result)
    (
        result = (x < low) ? low :
                (x > high) ? high :
                x;
        result;
    );

    function phasor(freq, reset) local(init, prev)
    (
        !init || reset ? (prev = 0; init = 1);

        phase = (freq / srate) + prev;
        phase = reset ? 0 : phase;
        phase = wrap(phase, 0, 1);
        
        prev = phase;
        phase;
    );

    function triangle(phase, p1)
    (
        phase = wrap(phase, 0., 1.);
        p1 = clip(p1, 0., 1.);
        out = (phase < p1) ?
            ((p1) ? phase/p1 : 0.)
        :
            ((p1==1.) ? phase : 1. - ((phase - p1) / (1. - p1)));

        out;
    );

    function slide(in, slideup, slidedown) local(init, prev)
    (
        !init ? (prev = in; init = 1);
        slideup = 1/max(slideup, 1);
        slidedown = 1/max(slidedown, 1);

        diff = in - prev;
        x = (in > prev) ? diff * slideup : diff * slidedown;
        x = x + prev;
        prev = x;
        x;
    );

    function t60(t)( pow(0.001, 1.0 / t) );

";

The source code for gen~ ops is available here and I directly copied a few of them. Would it be permissible to incorporate these functions into DynGen? Gen is a commercial product after all…

1 Like

(IANAL)

I couldn’t find a specific answer in the GPL FAQ.

Frequently Asked Questions about the GNU Licenses - GNU Project - Free Software Foundation – “free software with nonfree libraries” – is pretty close to the situation here. GPL doesn’t prohibit this, but does say that the GPL’ed software’s dependency on nonfree libraries would exclude it from free/libre environments. I think we have a pretty solid consensus in SC-land that we want to keep SC free/libre. In that sense, it’s probably not a good idea to lift copyrighted code verbatim.

Dietcv listed a few objects, and those objects’ specifications can be obtained from Max documentation and by observation of their behavior, without reference to source code. If somebody writes all-new implementations of those based on the specifications alone, then we aren’t stealing any code and there’s no copyright issue then.

hjh

As hjh mentioned, the license of the code is incompatible to the GPL license of SuperCollider, quoting from its header

The Software is licensed to Licensee only for non-commercial use.

and GPL does not limit its usage to non-commercial use. Therefore I won’t look further into this file in order to not get into any problems.

Yet it is totally fine to copy an existing interface - dietcvs answer is really helpful and I’ll start to implement those functions one by one, see Additional helper functions · Issue #30 · capital-G/DynGen · GitHub

3 Likes