My solution to the problem of file structure in SC is to not use multiple files at all. Each piece is one self-contained .scd file. Any common code is copy-pasted as boilerplate. No custom classes either.
As a programmer, the DRY instinct is to develop a reusable library. In practice, if you use a common library you’ll find yourself going back to the library to modify or add features, and find that all your other pieces that depend on the common code break unless you’re really cautious about backward compatibility. It’s one of the many ways that software engineering starts to creep into music-making, and suddenly you find yourself debugging a complex system and not making music anymore.
In general, I try to pursue radical simplicity in terms of SC code structure so I can laser-focus on patches and sequencing and not worry about software plumbing. I don’t find that this constraint has held me back creatively. Others may differ.
Quick shout-out / self-promo here for the Require quark, which provides some minor abstractions around loading other files. It can do relative path resolution as well as resolving against global “include” folders, supports wildcard resolution, and can avoid duplicate execution problems. Some of the path resolution stuff makes it useful as a standalone utility for locating other kinds of project files as well (I have a private sound file loading library that uses it to e.g. load a folder full of sound files).
Require("samples"); // loads "./samples.scd" or anything at (Require.roots +/+ "samples.scd"
Require("synths");
~player = Require("player"); // loaded files can return values
~player.play;
~attempts = List();
~drums = Require.resolvePaths(
identifier: "drum_*",
relativeTo: ["/Users/me/Documents/sounds", "/Volumes/Sounds"],
extensions: [ "wav", "aif", "aiff" ],
attempts: ~attempts
);
// returns files matching the pattern in the specified directories - if nothing was found,
// ~attempts will contain all the paths that were searched.
This is a really good meta-strategy: specifically, know what kinds of things you want to be working on and consciously adjust your practice to get there. In general, I think it’s worth choosing ahead of time what kind of commitment you want to have to solving “engineering” issues vs music making. This may be 0%, 10%, or 90% in one direction or the other… And then trying - as much as is possible - to hold yourself to that.
Depending on your personal make-up, engineering work OR music-making work may naturally bubble to the top. If it’s the former, you might find yourself writing libraries without ever making music. If the latter, you might find yourself fiddling with the same buggy, disorganized chunks code for weeks/months/forever where you might have just spent a few hours doing clean up or minor fixes.
I think it’s pretty rare to be able to make really good decisions about this while one is in the middle of a work session - personally, I tend to just follow whats in front of me at the time - so being conscious ahead-of-time about your strategy and holding yourself to it (e.g. what @nathan is talking about) is really important.
i think working with one specific method and refining its characteristics and adding and removing modules is pretty nice in finding out what a specific type of synthesis has to offer.
my goal always is to then interpolate between these different states found by discovery on this journey by refining things (using different modulators, sequences, parameter sets etc.) which is actually not really straight forward in SC to lay out a specific piece of music over time (micro, meso, macro structure) and I still havent found a solution which is suiting my needs in composing freely and get inspired by the things SC has to offer and is not lacking any complexity compared to a record and chop in the DAW approach. I often times think thats the reason why people stick with drone or looped based IDM (no formal evolution at all) or record and arrange things in the DAW (where i think that at the point where you record audio you are stuck with transforming your material). A filter sweep does not longer count to transform musical material.
I’d say the strongest arguments for or against SC system-building are found in anecdotal evidence.
In college I built a fairly elaborate live performance system in SC, and it was alright. But soon after I graduated, I dropped the idea of writing a layer between myself and SC and just wrote every piece as its own .scd file. As a result, my musical productivity skyrocketed, because I could make music without “being a programmer” and I could spend a lot more time on sound design and tangible musical materials. I’m currently happier than I’ve ever been with my SC workflow, and I’d also say my music got better but I’ll let you be the judge.
In contrast, I know multiple people who try to build systems in SC and hit some major walls. Some are technical, with sclang being a pretty shaky foundation for even medium-sized software. Some are more personal and motivational, where the artist underestimates the feedback loop between building an instrument and determining whether it’s creatively inspiring. Only a minority of them actually end up making and releasing music with their systems. I’ve been there: before I made a moderately successful system, I had several failures that had to be abandoned.
I’d be curious to hear any success stories of SC instrument design, e.g. @Sam_Pluta’s experiences. Also, while not an SC user, William Fields has put out some excellent music using his live algorithmic instrument, and his thoughts on that have been fun to read.
i think thats exactly the discussion which has to be made.
what is programming? what is contemporary music?
i went way to far on the programming part and got stuck with making music. but where do these meet?
i often get reminded when reading these threads that i have no idea of programming.
The advantage to system building is that you can build it up up over a long period of time. I have had a lot of success with this approach…and a lot of failure. The nice thing is I keep the successes and throw out the failures. I have some sound processing modules THAT I MADE OVER 20 YEARS AGO and still use in almost every performance today. I will probably make something next week that goes in the garbage next month.
This being said, I am not really making pieces. I am mostly an improviser who is building a language over time. My dissertation is called “Laptop Improvisation in a Multidimensional Space.” It isn’t about composition, as I decided a while back that the instrument, at least in my hands, is better at improvisation than at composition. My compositions tend to not be in the form of SC programs, although some of them are. I have found SC to be very bad for distribution of “pieces” even while it is great at composing.
The balance between creating good music and making good code is constant. SC, because of its idiosyncrasies, shall we say, tends to attract people who like to code and might find code attractive. Therefore we need to learn to see the forest through the trees. It is a constant struggle that we all need to be aware of.
I have some great success stories - I’ve built instruments over the last few years that felt at the time like way-too-big engineering nightmares, but after diligent attention and care (I’m talking a period of a month or two, not years…) they matured into very flexible, exciting things. I’ve built abstractions that let me work out ideas with a ton of fluidity and speed, and let me make great sounds all afternoon without any “engineering-y” work to be done at all.
But there are some caveats here:
I also have FOLDERS full of half-baked ideas, buggy implementations, and “musical systems” (though this particular kind of thing I, like nathan, gave up on years ago). The amount of wasted time is immense, though these days I tend to be quite selective about what I spend my time on engineering-wise.
I have 15+ years of professional software development experience, including large-scale work on systems… a fair amount more complex than any of my SuperCollider patches. This is a VERY particular and i think relatively unusual perspective to have in the SC community.
If I’m thinking of “success stories”, a few commonalities:
1. Small, simple, well-designed things that solve a specific problem I run into all the time.
The payoff with these can be fantastic, because the cognitive load of having to deal with some gotcha or clumsy way of doing something in sclang takes attention and energy out of every studio session. These are the equivalent of those hooks and fasteners you hang cables on in your studio - you shouldn’t spend ALL your time just re-cabling your setup, but if you trip over cords every day it’s worth spending a day or two solving the problem.
But, caveat: My ability to remember and use these nice little abstractions is limited: I probably have 15-20 of these in my core rotation, built up over many years. More than this and I simply won’t remember how they work, how to use them, or how to fix them if I find a problem. I have the energy and attention to care for a small menagerie of these animals, but definitely not a large one. And finally: your practice changes, so a useful thing now may not be useful in a few years.
I have have maybe 5-6 things like this. These took a LOT of time to build, and still take some effort to maintain. But, they are at a level of abstraction of a good VST plugin - they have reasonable parameters, flexible behavior, they sound good / work well, and I can throw them in something with no parameters or setup and they start working immediately.
All of these that I would consider successes started with code-jam style development - I’d see an opportunity that felt useful enough to really solve and not just hack around on. I’d spend a weekend or a week building a quick-and-dirty prototype, and make music with it. If it works well, sounds good - like really good, game-changer good - then I’ve made time to properly tackle the problem, make the code solid. This has meant REAL development: not just hacking around to make it a bit better, but really thinking about myself as the user, thinking about how I want to use it and what would make it better for me.
Usually when I reach a critical mass of “I really want it to do this!”, I’ll put in a well-defined chunk of work into it - e.g. a weekend or two - and then leave it alone for a while. The best things tend to grow like an onion, adding thing layers onto what was there before. I also have lots of things that never made it out of the first code jam phase, but these were often learning experiences - and one weekend for a failed experiment plus some learning is relatively low cost for me.
How big do your files get? I’ve always found that after 1000 or so things start to become unmanageable, particularly if the code needs to be executed it a particular order.
About 500 LOC for a standard 5-minute composition. I almost always have just two parenthesized blocks, one for SynthDefs and one for Routines, which take up about 50/50. Having many executable regions is a bad idea in any context other than live coding.
My code is not very software-y, it’s mostly data. The bulk of it is UGen graphs and manually sequenced rhythms and pitches.
Ah, unfortunately that simply isn’t possible for the genre of work I’ve been doing - Jellyfish 2023 - new work - it needs: osc sources between unity and mediapipe, several helper synths to wrap the mediapipe stuff, the audio, osc mapping functions, osc maps content, osc senders, and gui helpers for some of those things. Even if I simplified it there is easily several thousand lines.
Perhaps a one file approach works well for a particular style of music? And in general, if such considerations should be reflected in ‘opinionated advice for supercollider beginners’?
D’n’b producer Cylob used to have a big “do-everything” system in SC2.
Around 2005, I started writing all of my compositional code into Proto objects. Now here, I suppose saying this will trigger several standard rants about “prototypes are awful in SC” but I derive a lot of benefit from this. My live set up loads a lot of scd files but very, very few global variables (master and reverb mixer channels, a one-key hook for control mapping, and a tempo fader, that’s about it).
Now… this might not help beginners that much. But, for example, a few weeks ago when I was preparing for a duo jam, I wanted some vocals to adapt to the tempo and key. So I wrote a Proto process (which includes all buffer loading and unloading, mixing framework management etc.) and a factory object including a small database of audio files and metadata. Then, in the show:
// this loads everything -- \twist is the database key
/make(voxBP:tw(set:\twist));
// and this says "play a different section, on the downbeat, every 8 bars"
/tw = "1";
/tw..section = "[*]::\seq("1234")";
/tw = (main.rest*7);
It is more weight up front to put all of that into a Proto, but what I get out of that is a package that Just Works, and then I don’t have to fight with it again. 18 years of this working method (the live-coding dialect is more recent though), without significant design changes – I’m satisfied that this is working reasonably well.
The complexity level that Jordan is talking about would definitely benefit from objects, whether standard objects or the types of Proto that I’ve been using.
I absolutely needed to write classes to build the piece I’m working on now! Beyond that its just a joy when developer-me succeeds in providing user-me a pleasant interface! I spent a couple years using vanilla SC before venturing into writing my own classes, and it took a couple years of that before the system became (somewhat!) productive. I have my classes and pieces all in a giant git repo so if I change the classes one day and break old pieces it should be straightforward to wind things back (famous last words!).
Here ~a does not equal ~b because by the time the server has synced the current executing file has changed.
Now this just flat out doesn’t work with executeFile, which I think is better than sometimes working and sometimes not (just spent half an hour trying to fix this).
Yesterday I had said that nowExecutingPath might not be set in context of a thread. AFAICS that’s not true – whenever a thread is awakened, the thread’s executingPath populates nowExecutingPath, and when it yields, nowExecutingPath should be reverted.
The problem here is that load-ing a different file within a thread needs to update the thread’s executingPath, but it doesn’t. (I have a bit of a hacky fix, but it’s probably not a good one… will log it shortly.)
This probably explains the sync problem that you ran into.
I suppose that I can apologize for being unaware of that bug. However, I get a slight feeling that some of the energy in that post is going toward proving me wrong, where a better focus would be solving the problem… just noting it, I won’t go further into that.
The infrastructure is there for a thread to remember where it’s executing. It’s just not consistently updated, which IMO is not realizing the spec properly.
TBH here… in my live setup file, I have var path = thisProcess.nowExecutingPath and then I use path throughout. That was a while ago. I don’t remember if I had tried using thisProcess.nowExecutingPath, and it broke, and then I tried to work around it – or maybe I just assumed “this isn’t going to work in a thread” and preemptively worked around it.
(Now, the path infrastructure doesn’t exist for functions – so, scheduled functions don’t have any awareness of the path in which they were defined. I don’t think this is a big problem because standalone functions don’t persist their status in the way that routines do, and if you really need it, the workaround isn’t difficult.)