How to learn sound design

Creative Process
1

Continuing the discussion from Diving deeper into sound design:

Here’s a disorganized, caffeine-fueled wall of text on how I’d break down the acquisition of sound design abilities into stages. It’s about a fairly specific and narrow pipeline, although one that many SC users can relate to: making music by synthesizing everything from scratch. (Sampling and live instrument processing are entirely different crafts, ones I can’t seriously claim any qualifications in.)

This is based on my own experiences, and this linear process may not apply to everyone, it’s all subjective, etc. It’s biased towards the style of music I work in, which is experimental electronic music that straddles club influences and the academic computer music tradition.

As for my actual credentials, and why anyone would read my thoughts on this in the first place — I released an album made entirely with SuperCollider, no samples, external plugins, or libraries. If you don’t think my sound design is good, you can safely disregard what I’m writing here as braggadoccio. That said, a lot of what follows comes from talking to multiple friends who professionally design presets for plugin and synth companies.

Total beginner

This stage, you’ll mostly focus on following tutorials. Always learn from resources that have actual sounds (in particular not Gordon Reid’s/SoundOnSound’s Synth Secrets, which is overrated overall and not useful at this stage), and try to copy the sounds exactly as possible. Only copy sounds that you like in the sense that they get good results.

Invariably, pretty much any sound designer I know learns from YouTube. A simple first step is to just search there for “kick drum synthesis” (and scroll past the synthfluencers trying to sell you gear).

SeamlessR’s drum tutorials are probably the best resource to start with. Bunting is fantastic for learning bass sound design. They are both focused on dnb/dubstep/bass music but all that transfers well to experimental genres. Au5, Mr. Bill, Ned Rush also have some good stuff. All these people work with the standard DAW/VST setup. If you yourself work with a DAW/VST it’s critical to remake everything from scratch. Tune everything yourself, don’t just copy knob positions. If you’re using SC, then it is tempting to copy/modify other people’s code. Aside from the sketchy ethics of doing so, you won’t actually learn anything.

Even if you are solely interested in making electronic music without drums, you should definitely 100% learn how to make them. It’s fine to bias your tutorial intake towards your style, but also cover your bases.

Late beginner

You’re here when you’ve picked up the basic sound design formulas for EDM staples: drums, pads, keys, basses, bells. They don’t have to sound perfect, just recognizable, and you’re able to construct these patches from memory. At this point you’ve probably already started going off-page with the tutorials, getting less precise about following them and more creative about modifying them.

With the training wheels off, I still recommend that you work primarily in imitation and replication here. Find some tunes that seem within your reach, and make not just individual sounds but multi-layered arrangements, even if it’s just a 1-bar loop. A really important part of sound design is learning how multiple sounds in the mix interact and layer, and of course sound design and mixing are deeply intertwined.

Be aware of common mixing beginner mistakes — too much reverb and out-of-control high end are common beginner problems. SC users (including myself) often have issues with not enough reverb and excessive dynamic range.

When you do imitation exercises, also try to copy the mix. (Reference tracks are essential for mixing at any skill level.) There’s a lot of bad mixing advice out there that’s just holdovers from recorded music, and of limited relevance to people synthesizing everything from scratch. Really, in our case, 90% of mixing is just setting levels. If the levels don’t seem to work out, it might be a sound design issue.

Also at this point, I cannot stress enough how important it is to get a decent listening setup. One professor friend of mine tells their students in an SC class: “get yourself a $350 pair of studio headphones, that’s your textbook for this course.” Entry-level professional setups are pricey, and studio monitors and a good room especially so, but if you take sound design seriously, you really do not want to train your ears to compensate for cheap headphones. I made that mistake for far too long and my bass range is STILL too quiet to this day.

Intermediate

You’ve progressed to this stage if you can make most or all the club staple sounds (drum hits/pads/basses), and make them sound clearly professional and not-cheesy, and make such patches from scratch efficiently without the help of a tutorial or even a reference. If you’re starting to feel “tutorial fatigue,” i.e. you don’t really feel like you get much out of most YouTube sound design tutorials, then you’re probably here.

This stage is also where real original patches start emerging out of your speakers, as you start mixing and matching the various approaches you learn to synthesis. Of course, experimentation comes naturally at every skill level, but this is the point where it really starts accelerating your abilities, because you now have the taste and the ears to identify a successful experiment, and how to fix a failed one. Beginner experiments often fall into cliches, when you have experience backing your explorations then you’re much better at knowing what to avoid (reminder that this is all autobiographical).

When I was at this stage, most of the sound design tips I learned came from talking with colleagues and instructors at the CalArts music tech department, there are a lot of weird little secrets in sound design that have never really been published or written down and purely propagate among circles of friends. You don’t need to pay for art school to find these people and these ideas — forming a feedback group with fellow sound designers is an excellent way to build this kind of network. In general, feedback from serious musicians is really critical for building your ability at this stage and beyond.

Advanced

This is a stage where you can pretty easily go from imagining a sound in your head to realizing it as a patch, and do so quickly and with clean, professional results, so the route from creative inspiration to patch is as direct as it can get. Advanced sound designers have spent enough time with synthesis and effects that they can hear most synthesized sounds and pretty quickly think of a signal chain that could lead to it. Experimentation also comes naturally for them, and they have a good intuition about which experiments are more likely to succeed.

The roads diverge a lot here, since the upper echelons of sound design are really personal and different. I’m sure a lot of you read DSP research papers, Curtis Roads, etc but this stage is where the nerdiest computer music novelties help the most, when you know all the rules and now you’re marshalling the ammunition to blow up the rules and let the granular synthesis shrapnel shower down on your music.

Although theory can lead you to some really cool places, sound design should never be a purely theoretical pursuit. Keep doing replication exercises constantly, study the best sound designers you can find, challenge yourself with sounds that are particularly difficult to remake (including non-electronic sources), and whatever gets lost in transition is your creative contribution — if you’ve reached this stage then it’s hard to go wrong in that way.

Remember to keep your mixing abilities honed, get yourself a folder of reference tracks that you think are particularly great-sounding, and check against them often on multiple listening systems.

I would say I went from a total beginner, no experience with electronic music at all, to my current skill level in about 8 years or so. It was particularly in 2020-2022 when I started getting really serious about practicing sound design, maybe spending a few hours in SC every few days. I have said this many, many times before but learning sound synthesis is much like learning a traditional instrument. There are cerebral elements, but it’s also an intensely physical craft, all about setting knobs just right, all work that cannot be intellectualized or captured in papers. I am certain that if I got feedback more regularly my learning process would have been accelerated, but hindsight 20/20.

As always, keep in mind the only important principle: if it sounds good, it is good. I have wasted so many hours trying to squeeze good sounds out of an algorithm that looked cool but just wasn’t that musical. Anyway, I should stop blathering about this and go back to working in SuperCollider. Hope this helps in some way.

13 Likes
2

hey nathan, cool post. what was your background before that? I got the impression you already had some background in math or CS before?

This discussion brings an idea with it… Maybe we could organize a wiki/ repo/ book, but not just another supercollider tutorial, instead a cookbook of techniques with examples in supercollider, intermediate and beyond. Some planning and formatting guidelins, but open for community contributions. We have a lot of stuff out there.

1 Like
4

Me too, started writing a score editor in GW-BASIC around 13 or 14, at the time I already played the piano and had some simple pieces. Many years later I restarted with SC. I’m sure I still have some bad-habits from dealing with BASIC ahhah

Since we a sharing work in sc, this was all done with SC, all the sound synthesis (speakers and transducers in different objects) and also the written score: Bernardo Barros - Attrito [2018] | International Contemporary Ensemble feat. David Fulmer on Vimeo

1 Like
6

Great idea, @nathan let’s talk SOUND

7

check this out Jaap Vink Analogue Unstable Ring-Modulated Feedback

1 Like
8

this is a good advice!

10

I really appreciated this narrative and got a lot of thoughts bouncing around in my head after that. Thanks, Nathan, for taking the time to write it up.

My sound design problem is: I’m having trouble bridging the gap between “interesting” sound design (more complex instruments with more parameters and more modulation) and an improvisation interface that excels at rhythm and texture. Modulation works best with fewer, longer notes; rhythm tends to neutralize modulation so that you hear notes with varying parameter values per note, but movement within a note is harder to hear when the notes are relatively short.

So I can spend a lot of time “dressing up” a synthesis method with a lot of cool modulators and get a nice drone out of it – but then it just doesn’t translate into sequencing. (Also, as the instruments get more complex, they become a lot less practical to manage onstage – too many parameters means I run out of faders too fast, and too many little custom things for each instrument can cause the player process to die with an error if the slightest thing is done wrong.)

After a period of trying to make that work, I’m coming to accept that the modulation approach isn’t working for me. I’ll have to force myself to keep the notes simpler and focus on sequencing in the improv interface. But I’m also finding (I guess reflecting “intermediate” level synthesis) that, while I can do analog style, FM, wavetables, oscillator sync, resonators, etc in my sleep, I’m not quite satisfied with the results. They don’t sound bad, just lacking in character or individuality. (Aside: CPU heavy though they are, Mads’s ported plugins help with analog style a lot. I used to have this patch in Rack that I liked a lot, hooked it up by MIDI to my SC sequencer. The Vult Unstabile filter is so beautiful, made an otherwise dead simple patch sound special… switched to Cardinal running under VSTPlugin and haven’t found an equivalent filter, sad… anyway, ported plugins filters have some nice dirt in them, I should probably lean on them more.)

That "1+1=3 is something I haven’t dug into, really at all, and may be a way out of the “well… that’s another standard issue fm patch” dead end.

A significant deficiency for me is glitchy synthesis. I know low-frequency FM with feedback (though this can easily get locked into a cycle), otherwise not sure which direction to go. My sounds tend to be too tonal and I’d like to balance that out.

Alternately, I could self-impose a limitation and deliberately use only very simple sounds. That may be a bit of a cop-out but many artists get away with worse.

TIA,
hjh

1 Like
11

James, do you know the work of Peter Blasser? What you wrote about “glitchy synthesis” made me think of his work, somehow. There is a kind of simplicity in is highly unconventional synthesis circuits that I think would be an unique case, somehow. I was playing with his Deerhorn this week, and I was thinking about how this “highly unconventional simplicity” shows in his work (of course, it can be played in many different ways)

1 Like
12

I don’t, but I can look up some media. Thanks for the tip!

hjh

13

This video someone else explains this particular instrument (Deerhorn):

https://www.youtube.com/watch?v=FGYkUpCLdBs

But peter blasser has a lot of stuff. Hope it’s inspiring.

He’s a instrument designer, but musical ideas are there.

14

It’s interesting to think about live setups and how to get into new areas with them. I daydream that many buss of varying effects sends and smooth control over objects that act like clock dividers being a fun a way to change rhythm and textures live… but this is a daydream as I continue to learn.

15

I was once in your shoes not that long ago. I perused the SC resources and felt at loss for what to do with the software or what I was able to do since I had grown up with LMMS. It took a great mentor, Eli Fieldsteel, to help me connect the tools to the things I cared about. This is to say that no matter how many books and resources exist, I still feel that in the end, a lot of people could benefit from mentorship no matter their level of expertise, and with the disintegration of community and privatization of public spaces and the commodification of higher ed, that’s what’s responsible for this. And so, after Eli taught me, I started teaching others musical sound design. Students need to be shown how to apply the concepts they’re learning and why they matter. I feel that’s what you’ve been missing in your beginner sound adventures.

Another thing to remember is that this is still a developing field, despite having a history. I’m reading Chion’s Audio-Vision, which I’m loving, and there’s a section where he proposes the formulation of a theory on sound design which makes detailed sound descriptions possible, just so that sound design is something more teachable.

So I wish there was a better answer I could provide you. The technical side is rather straightforward. Pick up the Computer Music Tutorial by Curtis Roads second edition and Microsound and there you have all the compositional techniques and tools you’ll ever need.

The artistic side is still really obscure. A lot of prominent electronic musicians were and are secretive of their methods. Either for elitism or because they don’t know how to teach what they know and haven’t worked to translate it since it’s not their job or responsibility. The more likely reason is that it’s not really important “how” sound was created, but moreso “why”. Why did they make that sound design choice? What effect does it have?

The issue is that people need to be taught how to listen in the way that Albers taught us how to see using color interactions. That’s what Chion’s Audio-Vision is teaching me regarding film, what roles the sounds are playing and why the sound design choices made are being made. I didn’t have a framework to think about what I was absorbing beforehand. Even worse, my approach to music as an electronic musician and sound designer/artist in its orientation towards texture is seen as losing the forest for the trees by “real” musicians. We really need to be taught how to analyze and understand what we’re perceiving if we want to get anywhere as sound artists (I use that term loosely).

1 Like
16

Another concern is theft – James McCartney said once that SC2 users would sometimes send their work to him, and he recognized some code from the examples/ folder that hadn’t even been modified, just incorporated whole. If one has spent hours tuning a sound to be highly individual and articulated, one might not want that to be simply lifted for someone else’s work without permission. I personally don’t mind giving away some formulas (i.e. synthesis methods, like, this is how I do phase modulation etc.); whole SynthDefs are a bit harder to just put out there (though I did give away some, in my live coding system).

I find similarly in my teaching that students may be competent with VSTs while having zero conception of how they work, and a tendency to give up rapidly when they get into slightly unfamiliar territory (the “oh that’s math, I can’t do math” thing). Still trying to refine my approach to this. In SC, though, there are a lot fewer places to hide from the theory.

hjh

17

Another concern is theft – James McCartney said once that SC2 users would sometimes send their work to him, and he recognized some code from the examples/ folder that hadn’t even been modified, just incorporated whole. If one has spent hours tuning a sound to be highly individual and articulated, one might not want that to be simply lifted for someone else’s work without permission. I personally don’t mind giving away some formulas (i.e. synthesis methods, like, this is how I do phase modulation etc.); whole SynthDefs are a bit harder to just put out there (though I did give away some, in my live coding system).

I understand the concern, sound designers need to make money. But it doesn’t seem to me that guarding ones patches really prevents anyone from copying, it just slows down what’s inevitable, as piracy and reverse-engineering does with software. In my own practice, I rarely reuse sounds… because to me that would invalidate my purpose for making electronic music: to explore new sounds and epistemologies of sound.

To me, what makes electronic music so special is the ability to reproduce and study patches the way you would with source code. Every element is inspectable and analyzable. This is why I’ve always released my art and source code under CC BY-SA and GPLv3. I feel that if I didn’t do this, I’d be gatekeeping knowledge and discovery, keeping users in the trap that they need to buy another plugin or another piece of hardware.

I find similarly in my teaching that students may be competent with VSTs while having zero conception of how they work, and a tendency to give up rapidly when they get into slightly unfamiliar territory (the “oh that’s math, I can’t do math” thing). Still trying to refine my approach to this. In SC, though, there are a lot fewer places to hide from the theory.

Yeah, the math looked to scary growing up math illiterate, but then I realized how simple it was (though I later learned to find math interesting in itself).

Do we need an overhauling of the sound design pedagogy, and what would that look like? How do we teach people how to listen, how to think about sound synthesis, how to engage with electronic music and audiovisual works?