Proteus neural amp modeling UGen

Hi all,

I made an implementation of the Proteus neural net guitar amp/pedal modeler here:

Proteus

There is a universal mac build under releases. If someone wants to make a Windows build, I’d love to be able to upload one of those as well. It is a little tricky to build, but I gave nice instructions.

The UGen basically does what the Proteus VST does. It can load any neural net model trained for the Proteus VST plugin. I did not implement the cabinet emulation, but that is just an impulse response, so there are plenty of tools

The Ugen uses RTNeural under the hood, so it is super efficient (2% cpu on a 40 layer LSTM).

Let me know if there any issues or questions.

Sam

7 Likes

very very cool

What happens when you train with something different from a fx pedal? I imagine you can train with many things, I just have no idea how effective that is

It shouldn’t be hard to try. Anything that has no time element (no delays or reverbs for example) should kind of work. The website has pedals and amps. Any kind of waveshaper/distortion/etc should probably work. GuitarML has a Collab notebook for doing your own trainings and youtube videos showing how to do them.

Sam

Is that a backpropagation algorithm? If so, that explains the excellent performance. If that’s the case, it’s not the same technology that created the recent ´hype. (I’m not saying it’s better or worse, it’s probably better in practice)

thanks for sharing))

Yes. The LSTM network is trained using backpropagation.

Another reason it sounds great is the GuitarML guy spent 3 years figuring out the optimal network shape and training. A ton of work went into this. He started with WaveNet models, then RNNs, and settled on LSTM.

Sam

2 Likes

Sounds really interesting. I have been using the Neural Amp Modeler (NAM). I have made really nice models of my Fender Blues Deluxe Amp and also my Mud Honey distortion pedal. I am really pleased with the models, in general they are very faithful to the original and I doubt I would be able to tell them apart in a mix. If I played really hard on my guitar the dynamics are not exactly like playing the amp but still close. Did you try NAM, I am wondering how Proteus checks out against it? Does the Proteus let you create you own models?

Really cool

Do you know what tools most people use? Python libraries? Things like this? Graph neural networks in TensorFlow

Imagine training with your synthdef graphs and other things using this. (maybe someone is doing this already, I don’t follow everything )

I have also used NAM, but I don’t have a critical comparison of which is better.

Proteus does let you train your own models. The trainings are done with PyTorch.

RTNeural accepts trainings from TensorFlow and PyTorch. The next thing I am making is a general RTNeural loader. This won’t be as efficient as Proteus, but you will be able to load any model you train. Just a few wrinkles to iron out.

Sam

1 Like

Thanks @Sam_Pluta !

Very nice!

I just tested it, but when I load a model I get this error. Do you have any idea why?

Best,

José

ERROR: Message 'inputs' not understood.
RECEIVER:
   nil
ARGS:
CALL STACK:
	DoesNotUnderstandError:reportError
		arg this = <instance of DoesNotUnderstandError>
	Nil:handleError
		arg this = nil
		arg error = <instance of DoesNotUnderstandError>
	Thread:handleError
		arg this = <instance of Thread>
		arg error = <instance of DoesNotUnderstandError>
	Object:throw
		arg this = <instance of DoesNotUnderstandError>
	Object:doesNotUnderstand
		arg this = nil
		arg selector = 'inputs'
		arg args = [*0]
	SynthDesc:==
		arg this = <instance of SynthDesc>
		arg other = nil
	Object:!=
		arg this = <instance of SynthDesc>
		arg obj = nil
	Meta_Proteus:loadModel
		arg this = <instance of Meta_Proteus>
		arg synth = <instance of Synth>
		arg id = 'this_one'
		arg path = "/Users/josephfernandez/Downl..."
		var defName = 'proteus_test'
		var synthIndex = <instance of SynthDesc>
	Interpreter:interpretPrintCmdLine
		arg this = <instance of Interpreter>
		var res = nil
		var func = <instance of Function>
		var code = "Proteus.loadModel(~synth, 't..."
		var doc = nil
		var ideClass = <instance of Meta_ScIDE>
	Process:interpretPrintCmdLine
		arg this = <instance of Main>
^^ ERROR: Message 'inputs' not understood.
RECEIVER: nil

What version of SC are you using?

SuperCollider 3.14.0-dev MacOs 12.6 ARM

I can’t replicate this error. It also doesn’t really make sense. There is nothing called ‘inputs’ in the loadModel method. Can you send the code you are running?

This shouldn’t happen in vanilla SC – @Jose must have an extension overriding SynthDesc ==.

hjh

Thanks @jamshark70, @Sam_Pluta,

Is there any way to find out which SynthDef is causing the problem?
I just removed all the extensions, but the same error still appears…

Thanks,

José

I have no idea. It is probably a quark. This is in the language, not on the server.

Sam

Reading the stack trace: loadModel is calling != on a SynthDesc.

Source code:

	*loadModel {|synth, id, path|
		//get the index from SynthDescLib
		var defName = synth.defName.asSymbol;
		var synthIndex = SynthDescLib.global[defName];
		
		if (synthIndex != nil) ...

… indeed, it’s getting a SynthDesc out of SynthDescLib, and calling != on it.

In the vanilla class library, SynthDesc != Isn’t defined specifically – it calls back to Object !=, which is { arg obj; ^not(this == obj) } – this is where we get SynthDesc:== in the stack trace.

At this point, SynthDesc == should, in the vanilla class library, fall back to Object as well. Object == simply does an identity check – something === nil shouldn’t be a problem.

But if we see an actual SynthDesc:== in the stack trace, then it means that a specific SynthDesc:== method is defined somewhere. This is not defined in the main class library, so it must be an extension. And this extension must be assuming that other will always be a SynthDesc. This is not a safe assumption – an equality check must always be prepared for the “other” object to be a different class. That is, whichever extension this is, this method is not correctly written.

You can find out which extension by looking up the implementation of == for SynthDesc. Then log a bug with the extension author to ensure that == returns false if the other isn’t the right class.

BUT @Sam_Pluta could also fix it by using synthIndex.notNil instead of synthIndex != nil – isNil and notNil IMO would be recommended over == nil and != nil because, as this case shows, equality checks may be more fragile.

hjh

I changed it to use not.Nil instead of !=. Jose, can you redownload and try it again?

Sam

It works!

Many thanks to both of you!

José

Glad to hear that – but there is still the question of the extension with the faulty == operator.

What’s the result of this statement, in your environment? (With extensions enabled.)

SynthDesc.findRespondingMethodFor('==').filenameSymbol

This should give the location of the added == method.

hjh

Thanks, James,
The result is: /Users/jose/Library/Application Support/SuperCollider/downloaded-quarks/NodeSnapshot/NodeSnapshot.s, where there is an ‘inputs’ method.

Best,
José