SuperCollider 4: First Thoughts

I’d imagine the most learned/adopted language is JavaScript (and there’s TypeScript if you want types…) and JS has the advantage of running in browsers so there’s that…

see Gibber for a sc-like in-browser experience

(I am actually a fan of sclang but its the first and only language I’m ok at!)

1 Like

The slowest part of sclang BY FAR is the garbage collector. JVM’s realtime collector is likely to be an order of magnitude more efficient than sclang’s current Baker treadmill implementation, and would probably result in both better realtime guarantees AND better overall performance for the language.

There are already scsynth clients in other languages, as well as libraries for composition / event processing / etc - I don’t think any of these would really constitute a “SuperCollider 4” - but rather just “LuaCollider” or whatever. With a critical mass of development effort, I could imagine one of these could become sophisticated enough to transition the majority of the community to it, but I don’t know of any solutions that very close.

If “SuperCollider 4” has at least some continuity with CURRENT SuperCollider - e.g. it would be possible to transition some of the class library and port existing code, even if there wasn’t 100% backwards compatibility - the best possible path forward is using graalvm and truffle to build an sclang implementation against a JVM backend. The parser-compiler-interpreter part of this is probably not even a crazy-large amount of work (though it requires some programming language expertise that’s hard to come by…).

The much bigger chunk of work would be porting all of the existing primitives (many many hundreds?), which would need to be more or less rewritten from scratch. Because a JVM port of sclang would be actually-fast, it’s likely that almost all of the primitives could be rewritten directly in sclang code with no detriment to performance, which is of course pretty fantastic. But this is still a HUGE code surface area, and would probably require large scale community participation if it ever had a chance at being finished.

The upside here is that - if sclang was ported to JVM, it would have as good or better performance than most scripting languages (graal ports of languages tend to outperform their own VM’s). This would put it waaaaay out at the front of the pack of creative programming languages - if this vision of SuperCollider 4 actually succeeded, it probably would mean a HUGE expansion in the SC capabilities and community.

It’s a cool future to imagine… I guess we just need to find a programming language / VM expert and the funding to pay for a year or two of development costs so it actually gets finished. :slight_smile:

5 Likes

You’re probably right here - I think it’s possible to make a language that is both strongly typed and accessible for beginners. Processing is strongly typed, and is MUCH easier to learn for a newbie programmer than SC.

The problem is that SuperCollider / sclang, as it stands now, leans HEAVILY on it’s dynamic typing. For example, it’s very hard to imagine reproducing the features of the Event / Pattern system in a statically typed language. If you were to convert the implicit dependencies of this system into explicit interfaces that could be implemented by classes, I worry that it could result in a nightmarishly complex web of class / interface relationships…

A statically typed solution WOULD be useful and cool, but as per my previous comment - I’m not sure it we can call something “SuperCollider 4” if it ditches backwards compatibility and the majority of the concepts and patterns from SC3.

Why 4?

Revolutions allways fail. Rewrites are like revolutions (Perl 6?). Evolution is the way. If there’s something seriously wrong with what we have now, fix it. Re-evaluate and find the next bottleneck. Chain and weakest link et al. If there’s something superfuous, ignore it or delete it.

Using a general purpose language instead of a DSL will not work. Forget looking into other languages. Go to the butcher, look at his work shop. Go to the carpenter, look at the workshop. Go to the black smith … All very simmilar yet so different. Look at wood workers, making their own tools. Tools for their trade. Tools adapted to their habits. A cabinet makers plane is different from a table makers plane. These differences is what makes that you can’t hammer red hot steel in a butchers workshop.

In software, I use POV-Ray SDL, a DSL, a lot. It has its quircks. Looking at all the web 3d stuff in javascript is terrible. It is expressive is all the wrong ways. The same with Python layers over POV-Ray SDL. The same with python layers over the current SuperCollider. Or the other variants available.

So, what is the bottle neck that 4.0 solves?

4 Likes

A different way to look at it:
In programming, major version jumps (3 → 4) signify breaking changes - a lack of backwards compatibility. SuperCollider 4 means a SuperCollider where you may not be able to run your SC3 code, where you may need to rewrite some of it, or you may not get the same results.

Breaking backwards compatibility can be hugely disruptive for users - it has a cost. But, maintaining backwards compatibility is extraordinarily expensive for the development team, and makes certain kinds of changes so expensive or difficult that you can simply never do them at all. It’s expensive for users as well, because they may be stuck with archaic/old/weird/bad patterns that only exist because they are needed for compatibility. In this sense, a 4.0 can be less expensive than continuing to iterate on a 3.x - depending on what you want to achieve.

So, the question of a 4.0 is: if we’re going to incur the cost of a bunch of breaking changes to our code, what is the feature list that would be worth this cost?

2 Likes

There will always be old gits that prefer 1.5.2 :wink:
What I see in the discussion for example is using different languages. Splitting server side and client side “completely”. Different type systems, other VM’s etc.
Yet, even with significant changes, the first question imo still should be what are the real bottle necks? Does solving them need breaking changes? I can’t (and won’t) judge. If there are, go ahead. If it’s “just new chrome”, don’t.
The second question is for whom. The live coder, the casual user, the non real time user… Not every user group is as vocal, never forget the “silent majority”.

Some language-related feature requests are: stronger typing and step debugging.

Also there’s a known interpreter bug where try unwinds the stack incorrectly. (This is a very very nasty bug, unfixed for a couple years now, I think.)

AFAIK at present nobody in SC land knows enough about interpreter internals to address these.

Another way to address these problems would be to go to a language with some stronger typing features (though here scztt’s remarks about type flexibility in the pattern/event system are on point), and a step debugger, and where error handling “just works.”

Btw a glance at the elixir intro docs made me :exploding_head: and :drooling_face:

hjh

Just adding my two cents: if we were to switch languages/VMs, it would be great if it had a simple C interface, so people can easily integrate existing libraries. Currently, sclang only allows extensions written in sclang itself and is therefore very limited. It’s been more than once that I’ve hit a wall…Pd, on the other hand, allows you to interface with virtually anything.

1 Like

On top of my own wishlist:

  • Unify the callback mechanism by basing it entirely on Object:changed. (Rewrite NotificationCenter to use changed mechanism)
  • Simplify Responder Class Hierarchy, Refactor/cleanup CmdPeriod, but also OSCFunc, MIDIFunc possibly basing it on the Object:changed mechanism
  • Rewrite Server class, especially simplify it and get rid of many confusing variables and helper classes.
  • Refactor default parent Event. Simplify (or entirely get rid of) nested functions for playing event in Pbind, delegating them to Classes, while retaining or rather enhancing flexibility of behavior.
  • Incorporate an Event playing class which has access to its own keys while playing a stream and can stop and restart from the place in the streams where they stopped as well as modify the keys of its event while it is playing.
1 Like

Something else that struck me: someone suggested class namespaces, but what could be useful as well is a kind of distinction, either through a namespace or through some naming convention or some other mechanism, of what code is supposed to run server-side and what code is meant to be running language-side.

1 Like

My 2 cents: We’ve seen many such discussions over the years, but little consensus. As Scott points out, SomeOtherLangClientCollider has already happened more than once. Some of those live on, some have fizzled. To become a replacement, I think such a client would need to have significant advantages to both new and existing users. Otherwise SC3 would likely just continue and remain the primary concern with server development.

With that in mind Scott’s proposal here is one of the few I’ve seen that seems likely to have a real chance of success. I quite like the different angle!

1 Like

If we’re doing wish lists, one thing I’d like is to move the Node/Buffer/Bus allocators into the server. This would simplify a mountain of code and make multi-user situations much easier.

This wouldn’t have to wait for an SC4 though…

1 Like

I would really like to see the community organizing some crowdfunding to maintaning/test this idea. I honestly think this is a cool thing do at the present moment.

This also makes me think, aren’t we currently needing something like a foundation (like Processing and Blender have theirs) to organize, fund, support, promote etc SuperCollider ? Feasible ?

3 Likes

I wanted to know how the community felt about everything, and just what road is this heading towards… what goal, what end game.

SC4 is an inevitability… but there is still time to change the course of it’s destiny, I believe.

Crowdfunding is one manageable idea… if we can’t build ourselves to commit to the active development, we could contribute to the source of funding that would enable someone more experienced to do so, for us.

While payment can replace passion… if we’re going to pay someone to make a significant change, what we should be paying for, I believe, is a way to make passion a payment unto itself…

Hi there,

How many core developers are currently working on maintening Super Collider ?
I imagine its a lot of work to garanty SC stability, bug corrections, port it to new os and so on.

Then it could be a good thing to see SC supported by more institutions maybe in order to make the task easier to core dev. and give them space to make big changes, improvements… ?

Best,

Pragmatically – software maintenance is a lot of grunt work. Rewriting the interpreter for graalvm would bring benefits, but it isn’t glamorous or fun, and it’s more work than you’d think. In a perfect world, that work would be intrinsically rewarding but we don’t live in that world (and I’m also reminded of the way that capitalist ideology sometimes uses the idea of “intrinsic rewards” just to avoid facing the cost).

We have Qt GUIs and the IDE because a couple of programmers got a grant to fund their development – they could work basically full time on it for part of a year. Without that intensive effort, those projects could easily have simply not gotten off the ground. It’s hard to build momentum when you’re stealing a few hours a week for it.

I don’t think SC4 is inevitable. People have to do it, and have to have time to do it. Passion isn’t enough.

hjh

3 Likes

What I would like would be if the server was completely independent and modular. So if I want to embed it in a piece of software there is a clear C API (much like CSound) that I can use. If I want to set it up as a server there is a way for me to not only communicate through OSC, but also MIDI. And it should be possible to use the server in a standalone fashion, which would mean limited scripting (maybe LUA) in the server.

I agree.

…about how much are we talking, crowdfunding-wise ?

Can anyone ballpark the numbers

1 Like

And… how many core dev. do SC have currently ? Best,

1 Like

Hello everyone, this is a nice discussion to talk about the future of SC.

To recap, some common takeaways from this post are (feel free to add):

server:

  1. Allow users to define oversampling / undersampling multipliers for specific sections of a Node graph.
  2. Develop a way to have single-sample processing within a UGen.

language:

  1. Basically, port sclang’s features to a new language that would allow:
    a) step debugging.
    b) strong typing / type annotations, similarly to new versions of Python.
    c) easy C interoperability for easy development of language-side extensions.

To me, the improvements to the server would be the easier parts of this transition, as they do not require an entire rewrite of a programming language as the language issues would. Furthermore, once the server improvements have been implemented, they will be valid regardless of the language implementation that communicates with the server.

I would happily offer my help to develop the improvements to the server, as I have been working on similar issues in separate projects.

  1. Regarding the oversampling improvements, I have an idea on an implementation that would basically allow users to express oversampling multipliers at the Group / Node levels. This, however, would require the user to “split” his/her oversampled graphs from the rest of the code, leading to a situation of Groups within a Group. Alternatively, another idea is to leverage multi-channel expansion to develop the OverSample and DownSample UGens that can be used as sub-processes within a SynthDef graph, like so:
SynthDef(\oversampledTanh, {
    var oversample = OverSample4.ar(\in.ar(0));
    var tanh = oversample.tanh; //This would be 4 channels now, carrying the oversampled signal
    var downsample = DownSample4.ar(tanh);
    Out.ar(\out.ir(0), downsample);
}).add;
  1. Regarding the second issue, I am currently developing a JIT compilation engine for my audio DSL Omni. This engine could be used to offer ad-hoc UGens for single-sample processing in both scsynth and supernova. I already have developed the OmniCollider wrapper that allows users to compile Omni code to native SuperCollider UGens, and porting it over to the JIT engine would be very easy. My interface idea would be to treat the live-compilation engine just like Buffers: defining them on the language and having them allocated / compiled on the server. The process could be split in the OmniDef and Omni Classes. Then, after experimentations, users could easily compile their code to “proper” UGens by simply using the omnicollider command under the hood. I have worked on a similar interface in a previous project, JuliaCollider, which is now deprecated.
//Path to Omni code to "watch" for live JIT recompilation
~omniSine = OmniDef("~/code/Sine.omni"); 

//Using the specific OmniDef in a Synth. This will watch for code changes in Sine.omni to recompile in real-time
{ Omni.ar(~omniSine, 440) }.play

//Once happy with the implementation, compile to UGen
~omniSine.compileUGen;

Finally, I agre with @jamshark70 regarding fundings. Ideally, we all would love to work full time on our favourite piece of software, while practically, this is not possible. Having some forms of crowdfunding / organization backing the improvements to any work regarding SC would be the way to go in order for such changes to take place at a rapid pace.

3 Likes