LLM appreciation/rant thread

Hi everyone!

For a little over a year now, we’ve been watching the LLM revolution unfold, and recently it hit right at the heart of our community: the great James McCartney—the visionary genius mind behind the massive ecosystem that is SuperCollider—is using LLMs not just to port code, but to spin up entirely new languages and prototypes.

A good chunk of the people here work in other areas of tech, programming, and STEM, and we all know this seismic shift isn’t just hitting our turf: the debate over AI-generated code in the Linux kernel caused a massive uproar, and recent AI breakthroughs in advanced math and millennium prize problems even had 3Blue1Brown making a video asking, ‘Should you still pursue a math career?’

I’d really love to hear where everyone’s head is at right now, in what feels like the last remnants of our humanity. How is everyone digesting all of this?

I’ve been seeing devs from across the board dropping all sorts of takes. It ranges from full-on Luddite “smash the machines with a hammer” vibes, to total disillusionment like “I don’t write, review, or even look at code anymore; the magic is gone.” On the flip side, you have people who are hyped beyond belief because, for the first time, they can pull off hyper-advanced projects or finally break through the coding barrier that always held them back.

What does this mean for our community? Is writing SuperCollider code still a thing? Does making music even make sense anymore? What about making music through code? Is treating code as an artisanal craft for art’s sake our saving grace? Are we human after all?

I’ve been really concerned about the headspace of my closest colleagues lately. Let’s open the floor to everyone’s thoughts with zero judgment, and hopefully, this can be a place where we can all decompress a little.

Warm, human greetings.

4 Likes

To answer this interesting and thought-provoking question quickly and then get back to the most important task I should be working on, I wrote this post using the following process:

  1. I drafted it in my native language, Korean.
  2. I translated it using AI.
  3. I reviewed the translation and corrected it.
  4. I asked AI to review it again.
  5. I reviewed it once more myself.
  6. Finally, I submitted it here.

I tend to be interested in new technologies and tools, so I have been using AI fairly consistently over the past two years. My main use cases have been:

A. Proofreading English and German texts that I have written
B. Translating Korean texts that I have written into English or German
C. Writing educational materials for my students and academic texts
D. Creating diagrams and illustrations for those materials
(for example, digital audio signal flow diagrams, or cross-sectional illustrations of TRS and XLR connectors)
E. Creating images for project proposals and grant applications
F. Designing posters
G. Reviewing the efficiency of SuperCollider code that I have written
H. Extending the functionality of my SuperCollider code
I. Vibe coding with HTML, Processing, and SuperCollider

In the course of doing these things, I have almost never used the output generated by AI exactly as it was produced from a single prompt. Occasionally, I have shared AI-generated code as an example, and even when I explicitly stated that I had not tested the code myself, some people seemed to regard that as poor etiquette.

Overall, obtaining good results often requires investing two to three times as much time as it would take me to do the work on my own. That said, if I put in that level of effort, the final result is sometimes better than what I could have produced by myself.

On the other hand, asking AI to expand upon or edit texts and code that I have already written rarely produces satisfactory results, even when I invest a great deal of time. I have had almost no success with images or diagrams in particular.

Instead, I usually need to provide AI with a carefully organised set of keywords, ask it to produce a draft, revise it myself, identify problems in the output, request corrections, and repeat that process several times before I end up with something genuinely useful.

That said, AI is considerably faster than I am at reviewing budgets.

To be perfectly honest, I find it rather puzzling when I read posts claiming that AI allows people to save substantial amounts of time and effort while simultaneously producing high-quality results. In practice, even when talking to Google Gemini, I spend roughly three-quarters of the conversation explaining that it has misunderstood what I said, or pointing out precisely how it has misread the context and ended up producing a strange response. The same is largely true for the tasks I listed above.

I still do not know whether I am simply using AI incorrectly, or whether expectations surrounding AI have been exaggerated.

In any case, I have mixed and complicated feelings about all of this, but thank you for posting such an interesting question.

1 Like

The way I evolve a piece is through constant experimentation, and that requires good understanding of how the synth works. So using an LLM to generate it would be like using a player piano to do jazz improvisation. What is the point.

I tend to use LLMs for 2 things: as a better google and to generate boiler plate, or common code.

I find they’re really good for researching the kind of stuff that tends to be scattered in code repos, or lots of web pages and collating it. You can’t necessarily trust it, but it provides a good starting point that allows me to get sufficient intuition to move forward. I check everything (which I suspect is rare, based upon what I’ve seen professionally).

I recently needed an AGC for some compressors I’m using from Faust, and finding that kind of info is a pain. Claude was able to present the theory sufficiently well, and generated some code that I was able to check, then adapt pretty quickly.

I also use it for code generation sometimes if I have something that I’d typically try and find an example of on github/internet, but I find generation a bit unreliable and I don’t love reading other people’s code and once I have to start instructing it - it usually is quicker just to write the code myself. I do like the fact that you can ‘discuss’ the code with it though, even if the discussions can be a bit uneven.

In general I’m wary of LLMs for coding, because I think the economics of Claude make zero sense and I think for personal/open source stuff - the kind of code harnesses you get with Claude will cease to be an option once the VC money runs out. Plus I like programming.

3 Likes

I’m currently using ai to help rewrite the compiler. I wrote the lexer, parser and ir structure and got it declaring classes by hand. AI has been really useful at speed up boring ‘plumbing’ style code, which compilers have quite a lot of. Because it is using my data structures I can review this really easily. It has definitively improved the quality of the code because when I need to refactor something that is simple but boring, I will put it off, now the AI does it, and I can just check it has done it right. A good example of this was changing a size_t to a uin32_t, boring and you have to edit lots of code, but essentially trivial once you know it is possible.

I regularly have to highlight code it has made, ask why it did something and suggest an alternative, particularly when it comes to data structures where it doesn’t consider the likely number of elements.

It has been struggling to understand some of the subtleties in sclang but it does learn once you’ve corrected it.

Where it really shines is write quick throw-away debug programs, which it can one shot, and in debugging. It is excellent at performance testing and debugging as it can quickly grep around a stack trace far faster than I can read it.

2 Likes

I never expressed my personal thoughts on LLMs publicly – apart from some comments in GitHub discussion – but this thread might be a good opportunity. Warning: long and rambling post ahead!

Does making music even make sense anymore?

Yes, it does. More so than ever! People still practice their instruments, play and improvise together. I still write ensemble pieces and do my live shows. For me, music and art in general is about communication and connecting with other people.

What about making music through code? Is treating code as an artisanal craft for art’s sake our saving grace? Are we human after all?

Code is a precise form of expressing your ideas. If you are designing new sounds or musical algorithms, you eventually need to understand what your code is doing to make small targeted changes. When I write an ensemble piece, I carefully notate every instrument. When I design the electronics, I write the code and tweak the parameters.

If you have a rough idea of what you want, you could make a prototype with an LLM and work from there. It’s not something I would personally do, but I can see how it might work for some people.


Then there is the whole philosophical, aesthetic and political angle. Personally, I try not to rely on tools that are owned by large megacorps. After all, that’s why I’m using open source tools in the first place! In the last decades, software developers have been in the privileged position to own the means of production, at least on the software side. There are free and open source operating systems, compilers, libraries, browsers and code editors. Honestly, I find it crazy that so many people are now becoming fully reliant on frontier LLMs. Many, if not most new software developers will simply not be able to write code without a subscription from a large AI company.

Don’t even get me started on the rising hardware prices that make personal computing less and less affordable for the masses. Or the environmental concerns. Or how unregulated AI progress affects society as a whole. We haven’t even learned how to properly deal with Web 2.0 and Social Media. Now we are supposed to handle an AI revolution? If we look at the people behind these companies, are these the ones we want in control of our future? Haven’t we learned our lessons?


I have to admit that beyond the broader political points, the recent advancement of coding agents also require me to (re)evaluate my position as an (open source) software developer and creative coder. The reality is that all the software I’m writing can be written by LLMs. I took some pride in being able to write good, efficient and reliable software. I don’t know how much of this pride is still left…

I sometimes ask myself why I don’t just use coding agents. Why go to the troubles of writing everything by hand? Then I’m reminded why I’m writing code in the first place: I’ve always had a desire to pull back the curtain and look at how things work. I wanted to build things from the ground up. (Of course, there is always a limit. I don’t write everything in raw assembly. I don’t write my own C++ compiler or build my own computers. I do use libraries and frameworks where it makes sense.) I get tremendous intellectual satisfaction from implementing algorithms, optimizing code or finding elegant solutions to hard problems. Asking an agent to write the code for me would take away a lot of the fun.

I think if you are only interested in the outcome, and not so much in the building process, then LLMs are really enticing because you can build so much faster, there’s no question about it. Of course, if you actually care about the quality, you will eventually need to slow down again. The thing is: if you really read and think about the generated code, you might often be quicker writing it yourself, especially for small targeted changes and in areas where you’re already proficient. Boilerplate, large refactorings and throw-away code are certainly a good job for an LLM.

I think learning to code still matters. How can you possibly evaluate the output of a coding agent and make informed decisions if you don’t know how to code? Some companies apparently came to the conclusion that they just don’t read the code anymore, which to me just sounds ridiculous. I recently watched an interesting interview with Mario Zechner (fellow Austrian and creator of the Pi coding agent) who has similar concerns (https://www.youtube.com/watch?v=GhjU-KvXtT0).

I am an artist first foremost, so it’s not surprising that I think more like an artist than a software engineer. Composers, painters and writers like to obsess over small details in their compositions, paintings or stories and not think too much about the time economics. It’s also about the feeling of authorship and ownerhip. In a similar vein, I don’t want to use AI to create the graphic assets for my game engine, I rather do the pixel art by hand. Yes, it takes more time, but it’s also fun, the end result will be more personal and I feel accomplishment. That being said, writing code can be a huge time sink and maybe I should make more art instead… Would coding agents help me with that? Or will I just end up writing more software? :smiley:

Personally, I do see code as a form of human expression. I want the open source projects I’m involved in to stay readable and comprehensible for other humans. This might sound a bit lofty, but I feel a kind of historical responsibility when working on software that is 20 or 30 years old, that’s why I’m hesitant about dumping LLM output into a project. I have seen some LLM-generated PRs and almost all of the time they were not the best solution to the problem, mostly because they miss the broader context. Sometimes they would have made the code worse. In a business scenario, you only need code that is good enough to do the job, there are typically no resources for crafting the best possible code. (Of course, this has been the case long before AI.) In an open source project, the only restriction is the time I’m willing to invest.

At this point, I need to make it clear that I do occasionally use LLMs. I treat them as better search machines and as a stack overflow replacement. It can also be extremely helpful for debugging obscure issues. (Often enough they can also put you on the completely wrong track.) In the future, I might also consider it for “boring” tasks such refactoring or writing tests.

The software industry has definitly changed and there’s no going back to where it was. Coding agents will play an important role in the foreseeable future. At the same time, we’re definitely in a bubble and the pendulum will swing back to some extend once the hugely subsidized subscriptions come to an end. I never worked in the software industry (and probably never will) so in the end coding agents don’t affect me all to much. The effect of LLMs on society as a whole, that’s another story…

8 Likes

A good example of this was changing a size_t to a uin32_t, boring and you have to edit lots of code, but essentially trivial once you know it is possible.

I’ve seen examples like this come up a lot, and I don’t quite understand why I’d need an LLM for this. I’ve had search and replace tools in my editor that can do this for 20+ years. Or does it do more than this?

A naive search and replace in a C++ code base often is a recipe for disaster.

I’ve done this for prototyping when there was some synthesis technique I didn’t understand, and I figured that seeing it in SuperCollider (or Faust code) would clarify it for me. But this is purely about understanding. I’m probably not going to use the code (or if I do, it will be heavily modified - the same way I might start from an example in a blog post, paper, or book).

The thing is: if you really read and think about the generated code, you might often be quicker writing it yourself, especially for small targeted changes and in areas where you’re already proficient.

This has been my experience. It also results in worse code. Not so much because of limitations in LLMs (though that is an issue), but because coding is how I think about code - if I’m not coding, my thinking is far shallower. I’ve had musical ideas emerge through coding. On the other hand, a lot of coding is kind of meh - and is essentially working from a template (a lot, and I mean A LOT, or web and corporate programming is basically that - and I think that gets left out of the debate).

1 Like

Which is why I wouldn’t do that, but also would be very wary of an LLM doing a naive search and replace. Presumably it’s doing more than that, hence the question.

Say you have this, and want to make it a std::unt32_t.

struct T {
   std::size_t a, b;
};

But you have a bunch of functions like this

std::size_t do_something_and_access_t(T, ... other stuff);
T construct_a_t(...a bunch of arguments, std::size_t will_become_the_a);
void foo (T t) {
   const std::size_t a = t.a; // this one could be auto.
} 

It will (reasonably reliably) turn places where we semantically have a t.a into std::uint32_t, not just syntactically. It lets you change every thing that means T:a.

Of course you need to review these changes.

2 Likes

I have always tried to write code that lets me read and write music - the challenge has been to determine “what do I want to write in order to express --musical idea–” and “how should a REPL-like process be designed that lets me build up a piece”.

So using LLMs is natural. The work of answering those questions, building a UI for myself to code in, is just as challenging. Development just goes much faster, hopefully allowing me more time to do the actual music writing (lol).

There are new challenges though! Cognitive overload! I’ve been building a little algebra of tempo maps - I can hardly remember the whole zoo and all the operations that were added! – I can go much farther down a wrong path than before!

I did enjoy little elegances and knowing how things worked (to a modest extent anyhow!) - I know much less suddenly - a little like wandering in a fog - but this can be thrilling in its own way.

We did come to these machines because we see creative possibilities in exactly “automation” after all… terrifying though this level of automation may be. The models are so much better than they were a year ago, things may become quite strange very soon.

3 Likes

I discussed such issues last week with students in the first term of our computer music study. Obviously, everyone must rethink the subject: developers, artists, teachers, and students.

IMO, the “death of coding” is only half the truth. First and foremost, it’s a welcome marketing argument for companies. As @cian points out, much real-life coding is a chore, especially when addressing clearly defined tasks. AI is a relief in this respect, and I totally understand the point of saving time and effort. James McCartney’s port of Synth-O-Matic to a modern MacOS is a breathtaking example!

But coding concrete sound and music is mostly not a clearly defined task: ideas come along the way, by experimenting, diving into the details, and discovering their musical consequences – a laborious but empowering feedback process. Cutting it off removes the edge!

On the other hand, AI will increasingly be part of artistic feedback processes. We all have to learn, experiment, and digest how this could work.

I urge students to become good at programming and smart in using AI! But also, without aesthetic reflection, all the amazing possibilities remain void.

All in all, I feel quite positive about our field from the perspective of artistic freedom. From a didactic point of view, I see new challenges: it’s increasingly difficult to convince students to spend a lot of time thinking through programming issues and practice coding. And I totally understand: why do so instead of just asking AI? Humans always tend to go the way of least resistance.

In my observation, AI and the collection of information via the internet can lead to a bifurcation of knowledge and capability. Some students can really boost their toolkit while others are overwhelmed. Not knowing what to do with a chunk of AI-generated code is just an example of such a situation. I see it so often, but there’s a way out by reflecting the causes: Recently, in our synthesis course, one student described the annoyance of such an experience. It triggered his desire to learn more about coding.

2 Likes

Thanks, you’ve expressed this more concisely than I ever could!

I want to add that this can also be true for non-musical software tasks. I’m currently (manually) reworking large parts of scsynth and supernova and while doing so, I keep getting more and more ideas. Not to mention how much I learn about the code base, particularly about corners that I’ve never looked into very deeply. All of this knowledge is important for making informed decisions. I had to shut down quite a few AI-generated/assisted PRs because they missed important context.

That being said, things like the Synth-O-Matic port are indeed a very good task for an LLM. Especially if the stakes are not very high.

I think it would be really interesting if you described how scsynth was supposed to work and then asked AI for any ways it deviated and any bugs it could find. Then compared what you found with what it found.