Neural network synthesizer modeling

Questions
1

Hi,

I am seeking out some assistance in improving the modeling of a neural network synthesizer.

Its structure is quite similar to what is described in Nathan Ho’s feedback integrator network blog post, and I took some inspiration from it and what is done in a Ugen found in sc3-plugins called Dneuromodule (quite unstable, providing a larger input will make the server crash) to implement a basic version of it, but I don’t fully grasp the math and non linearities and how they should be modeled (for now all I did was put some noise in the inital values and apply a tanh at the outputs of the neurons)

The sound is nothing special though for now, I think I expected something more like a No Input Mixing Board/ Matrix Mixer type of sound. Changing the weights and bias does not change the sound much.

How could one improve the modeling to make it closer to what you can hear in David Tudor’s Neural Synthesis album?
I think feedbacked matrix mixer patches accomplishes that as well, but then what would be the point in an architecture involving neural networks?

Server.default.options.blockSize = 1;
s.reboot;

// interface inspired from Dneuromodule UGen
// theta = bias for each node, x = initial input for each node
(
SynthDef(\nmA_native, { |out = 0,
                        x0 = 0.1, x1 = 0.3,
                        w0 = -3, w1 = 2, w2 = -2, w3 = 0|
    var y, nxt, sig, env, w, theta;

    w     = [[w0, w1], [w2, w3]];
    theta = [MouseX.kr(-2, 2), MouseY.kr(-2, 2)];

    y   = LocalIn.ar(2, [x0 + LFNoise1.ar(0.3), x1 + LFNoise1.ar(0.3)]);
    nxt = (w.collect { |row, i| (row * y).sum + theta[i] }).tanh;
    LocalOut.ar(nxt);

    Out.ar(out, Limiter.ar(LeakDC.ar(nxt * 0.1), 0.9));
}).add;
)

a = NodeProxy.audio(s, 2);
a.source = \nmA_native;
a.play;
a.gui;
a.free;
1 Like
2

I would play around with add some random fluctuation, noise. Assuming you want something more unpredictable. No guarantee it will work, but I usually find something interesting emerges.

3

First, you’ll want a lot more than two neurons to get interesting behavior. More importantly, simple, generic NN architectures like this one aren’t very musical if you set weights manually or randomly. At best you’ll get textured noise.

The term “neuron” can mean basically anything that takes in inputs and turns them into outputs with some kind of parameters. It’s perfectly legal for neurons to contain processing much more complex than weighted sum + sigmoid nonlinearity. For example this paper has examples like putting filters in the loop. So in a way, patching together synthesis units in matrix feedback is already a neural network, just a very specific one which happens to have a lot of sweet spots. A generic, textbook NN architecture used as a synthesizer may also have sweet spots (if it is large enough), but they can only really be found using training methods.

1 Like
4

The Dr Dobbs article author mentions the weights were set manually (only 2 signed voltage values) and that in the case of Tudor’s instruments, training was not used, although it could be.

The 2 neuron case with Dneuromodule seemed to exhibit some sort of chaotic behavior, according to the white paper that inspired it.
I will admit that the example in the schelp file is not very interesting.

I am aware of the RAVE project for NN-based synthesis, but I guess I was looking for something less heavy and wanted to prototype something lighter in SC, albeit inefficient.

I shall start looking at a matrix mixer architecture with customizable synthesis nodes…sounds like a job for the JITLib.

5

You might also check out @Luc_Doebereiner’s AutoRegNN:

https://git.iem.at/lucdoebereiner/autoregnn

It makes cools sounds.

The link doesn’t seem to be working for me, maybe Luc knows if the project is still active.

1 Like
6

I changed the name and moved the repo a while ago. You can find it here: GitHub - lucdoebereiner/ARMLP: Autoregressive Multi-Layer Perceptron - A neural network UGen for experimental sound synthesis in SuperCollider. · GitHub

2 Likes