LLM appreciation/rant thread

Hi everyone!

For a little over a year now, we’ve been watching the LLM revolution unfold, and recently it hit right at the heart of our community: the great James McCartney—the visionary genius mind behind the massive ecosystem that is SuperCollider—is using LLMs not just to port code, but to spin up entirely new languages and prototypes.

A good chunk of the people here work in other areas of tech, programming, and STEM, and we all know this seismic shift isn’t just hitting our turf: the debate over AI-generated code in the Linux kernel caused a massive uproar, and recent AI breakthroughs in advanced math and millennium prize problems even had 3Blue1Brown making a video asking, ‘Should you still pursue a math career?’

I’d really love to hear where everyone’s head is at right now, in what feels like the last remnants of our humanity. How is everyone digesting all of this?

I’ve been seeing devs from across the board dropping all sorts of takes. It ranges from full-on Luddite “smash the machines with a hammer” vibes, to total disillusionment like “I don’t write, review, or even look at code anymore; the magic is gone.” On the flip side, you have people who are hyped beyond belief because, for the first time, they can pull off hyper-advanced projects or finally break through the coding barrier that always held them back.

What does this mean for our community? Is writing SuperCollider code still a thing? Does making music even make sense anymore? What about making music through code? Is treating code as an artisanal craft for art’s sake our saving grace? Are we human after all?

I’ve been really concerned about the headspace of my closest colleagues lately. Let’s open the floor to everyone’s thoughts with zero judgment, and hopefully, this can be a place where we can all decompress a little.

Warm, human greetings.

9 Likes

To answer this interesting and thought-provoking question quickly and then get back to the most important task I should be working on, I wrote this post using the following process:

  1. I drafted it in my native language, Korean.
  2. I translated it using AI.
  3. I reviewed the translation and corrected it.
  4. I asked AI to review it again.
  5. I reviewed it once more myself.
  6. Finally, I submitted it here.

I tend to be interested in new technologies and tools, so I have been using AI fairly consistently over the past two years. My main use cases have been:

A. Proofreading English and German texts that I have written
B. Translating Korean texts that I have written into English or German
C. Writing educational materials for my students and academic texts
D. Creating diagrams and illustrations for those materials
(for example, digital audio signal flow diagrams, or cross-sectional illustrations of TRS and XLR connectors)
E. Creating images for project proposals and grant applications
F. Designing posters
G. Reviewing the efficiency of SuperCollider code that I have written
H. Extending the functionality of my SuperCollider code
I. Vibe coding with HTML, Processing, and SuperCollider

In the course of doing these things, I have almost never used the output generated by AI exactly as it was produced from a single prompt. Occasionally, I have shared AI-generated code as an example, and even when I explicitly stated that I had not tested the code myself, some people seemed to regard that as poor etiquette.

Overall, obtaining good results often requires investing two to three times as much time as it would take me to do the work on my own. That said, if I put in that level of effort, the final result is sometimes better than what I could have produced by myself.

On the other hand, asking AI to expand upon or edit texts and code that I have already written rarely produces satisfactory results, even when I invest a great deal of time. I have had almost no success with images or diagrams in particular.

Instead, I usually need to provide AI with a carefully organised set of keywords, ask it to produce a draft, revise it myself, identify problems in the output, request corrections, and repeat that process several times before I end up with something genuinely useful.

That said, AI is considerably faster than I am at reviewing budgets.

To be perfectly honest, I find it rather puzzling when I read posts claiming that AI allows people to save substantial amounts of time and effort while simultaneously producing high-quality results. In practice, even when talking to Google Gemini, I spend roughly three-quarters of the conversation explaining that it has misunderstood what I said, or pointing out precisely how it has misread the context and ended up producing a strange response. The same is largely true for the tasks I listed above.

I still do not know whether I am simply using AI incorrectly, or whether expectations surrounding AI have been exaggerated.

In any case, I have mixed and complicated feelings about all of this, but thank you for posting such an interesting question.

2 Likes

The way I evolve a piece is through constant experimentation, and that requires good understanding of how the synth works. So using an LLM to generate it would be like using a player piano to do jazz improvisation. What is the point.

I tend to use LLMs for 2 things: as a better google and to generate boiler plate, or common code.

I find they’re really good for researching the kind of stuff that tends to be scattered in code repos, or lots of web pages and collating it. You can’t necessarily trust it, but it provides a good starting point that allows me to get sufficient intuition to move forward. I check everything (which I suspect is rare, based upon what I’ve seen professionally).

I recently needed an AGC for some compressors I’m using from Faust, and finding that kind of info is a pain. Claude was able to present the theory sufficiently well, and generated some code that I was able to check, then adapt pretty quickly.

I also use it for code generation sometimes if I have something that I’d typically try and find an example of on github/internet, but I find generation a bit unreliable and I don’t love reading other people’s code and once I have to start instructing it - it usually is quicker just to write the code myself. I do like the fact that you can ‘discuss’ the code with it though, even if the discussions can be a bit uneven.

In general I’m wary of LLMs for coding, because I think the economics of Claude make zero sense and I think for personal/open source stuff - the kind of code harnesses you get with Claude will cease to be an option once the VC money runs out. Plus I like programming.

5 Likes

I’m currently using ai to help rewrite the compiler. I wrote the lexer, parser and ir structure and got it declaring classes by hand. AI has been really useful at speed up boring ‘plumbing’ style code, which compilers have quite a lot of. Because it is using my data structures I can review this really easily. It has definitively improved the quality of the code because when I need to refactor something that is simple but boring, I will put it off, now the AI does it, and I can just check it has done it right. A good example of this was changing a size_t to a uin32_t, boring and you have to edit lots of code, but essentially trivial once you know it is possible.

I regularly have to highlight code it has made, ask why it did something and suggest an alternative, particularly when it comes to data structures where it doesn’t consider the likely number of elements.

It has been struggling to understand some of the subtleties in sclang but it does learn once you’ve corrected it.

Where it really shines is write quick throw-away debug programs, which it can one shot, and in debugging. It is excellent at performance testing and debugging as it can quickly grep around a stack trace far faster than I can read it.

2 Likes

I never expressed my personal thoughts on LLMs publicly – apart from some comments in GitHub discussion – but this thread might be a good opportunity. Warning: long and rambling post ahead!

Does making music even make sense anymore?

Yes, it does. More so than ever! People still practice their instruments, play and improvise together. I still write ensemble pieces and do my live shows. For me, music and art in general is about communication and connecting with other people.

What about making music through code? Is treating code as an artisanal craft for art’s sake our saving grace? Are we human after all?

Code is a precise form of expressing your ideas. If you are designing new sounds or musical algorithms, you eventually need to understand what your code is doing to make small targeted changes. When I write an ensemble piece, I carefully notate every instrument. When I design the electronics, I write the code and tweak the parameters.

If you have a rough idea of what you want, you could make a prototype with an LLM and work from there. It’s not something I would personally do, but I can see how it might work for some people.


Then there is the whole philosophical, aesthetic and political angle. Personally, I try not to rely on tools that are owned by large megacorps. After all, that’s why I’m using open source tools in the first place! In the last decades, software developers have been in the privileged position to own the means of production, at least on the software side. There are free and open source operating systems, compilers, libraries, browsers and code editors. Honestly, I find it crazy that so many people are now becoming fully reliant on frontier LLMs. Many, if not most new software developers will simply not be able to write code without a subscription from a large AI company.

Don’t even get me started on the rising hardware prices that make personal computing less and less affordable for the masses. Or the environmental concerns. Or how unregulated AI progress affects society as a whole. We haven’t even learned how to properly deal with Web 2.0 and Social Media. Now we are supposed to handle an AI revolution? If we look at the people behind these companies, are these the ones we want in control of our future? Haven’t we learned our lessons?


I have to admit that beyond the broader political points, the recent advancement of coding agents also require me to (re)evaluate my position as an (open source) software developer and creative coder. The reality is that all the software I’m writing can be written by LLMs. I took some pride in being able to write good, efficient and reliable software. I don’t know how much of this pride is still left…

I sometimes ask myself why I don’t just use coding agents. Why go to the troubles of writing everything by hand? Then I’m reminded why I’m writing code in the first place: I’ve always had a desire to pull back the curtain and look at how things work. I wanted to build things from the ground up. (Of course, there is always a limit. I don’t write everything in raw assembly. I don’t write my own C++ compiler or build my own computers. I do use libraries and frameworks where it makes sense.) I get tremendous intellectual satisfaction from implementing algorithms, optimizing code or finding elegant solutions to hard problems. Asking an agent to write the code for me would take away a lot of the fun.

I think if you are only interested in the outcome, and not so much in the building process, then LLMs are really enticing because you can build so much faster, there’s no question about it. Of course, if you actually care about the quality, you will eventually need to slow down again. The thing is: if you really read and think about the generated code, you might often be quicker writing it yourself, especially for small targeted changes and in areas where you’re already proficient. Boilerplate, large refactorings and throw-away code are certainly a good job for an LLM.

I think learning to code still matters. How can you possibly evaluate the output of a coding agent and make informed decisions if you don’t know how to code? Some companies apparently came to the conclusion that they just don’t read the code anymore, which to me just sounds ridiculous. I recently watched an interesting interview with Mario Zechner (fellow Austrian and creator of the Pi coding agent) who has similar concerns (https://www.youtube.com/watch?v=GhjU-KvXtT0).

I am an artist first foremost, so it’s not surprising that I think more like an artist than a software engineer. Composers, painters and writers like to obsess over small details in their compositions, paintings or stories and not think too much about the time economics. It’s also about the feeling of authorship and ownerhip. In a similar vein, I don’t want to use AI to create the graphic assets for my game engine, I rather do the pixel art by hand. Yes, it takes more time, but it’s also fun, the end result will be more personal and I feel accomplishment. That being said, writing code can be a huge time sink and maybe I should make more art instead… Would coding agents help me with that? Or will I just end up writing more software? :smiley:

Personally, I do see code as a form of human expression. I want the open source projects I’m involved in to stay readable and comprehensible for other humans. This might sound a bit lofty, but I feel a kind of historical responsibility when working on software that is 20 or 30 years old, that’s why I’m hesitant about dumping LLM output into a project. I have seen some LLM-generated PRs and almost all of the time they were not the best solution to the problem, mostly because they miss the broader context. Sometimes they would have made the code worse. In a business scenario, you only need code that is good enough to do the job, there are typically no resources for crafting the best possible code. (Of course, this has been the case long before AI.) In an open source project, the only restriction is the time I’m willing to invest.

At this point, I need to make it clear that I do occasionally use LLMs. I treat them as better search machines and as a stack overflow replacement. It can also be extremely helpful for debugging obscure issues. (Often enough they can also put you on the completely wrong track.) In the future, I might also consider it for “boring” tasks such refactoring or writing tests.

The software industry has definitly changed and there’s no going back to where it was. Coding agents will play an important role in the foreseeable future. At the same time, we’re definitely in a bubble and the pendulum will swing back to some extend once the hugely subsidized subscriptions come to an end. I never worked in the software industry (and probably never will) so in the end coding agents don’t affect me all to much. The effect of LLMs on society as a whole, that’s another story…

11 Likes

A good example of this was changing a size_t to a uin32_t, boring and you have to edit lots of code, but essentially trivial once you know it is possible.

I’ve seen examples like this come up a lot, and I don’t quite understand why I’d need an LLM for this. I’ve had search and replace tools in my editor that can do this for 20+ years. Or does it do more than this?

3 Likes

A naive search and replace in a C++ code base often is a recipe for disaster.

I’ve done this for prototyping when there was some synthesis technique I didn’t understand, and I figured that seeing it in SuperCollider (or Faust code) would clarify it for me. But this is purely about understanding. I’m probably not going to use the code (or if I do, it will be heavily modified - the same way I might start from an example in a blog post, paper, or book).

The thing is: if you really read and think about the generated code, you might often be quicker writing it yourself, especially for small targeted changes and in areas where you’re already proficient.

This has been my experience. It also results in worse code. Not so much because of limitations in LLMs (though that is an issue), but because coding is how I think about code - if I’m not coding, my thinking is far shallower. I’ve had musical ideas emerge through coding. On the other hand, a lot of coding is kind of meh - and is essentially working from a template (a lot, and I mean A LOT, or web and corporate programming is basically that - and I think that gets left out of the debate).

1 Like

Which is why I wouldn’t do that, but also would be very wary of an LLM doing a naive search and replace. Presumably it’s doing more than that, hence the question.

Say you have this, and want to make it a std::unt32_t.

struct T {
   std::size_t a, b;
};

But you have a bunch of functions like this

std::size_t do_something_and_access_t(T, ... other stuff);
T construct_a_t(...a bunch of arguments, std::size_t will_become_the_a);
void foo (T t) {
   const std::size_t a = t.a; // this one could be auto.
} 

It will (reasonably reliably) turn places where we semantically have a t.a into std::uint32_t, not just syntactically. It lets you change every thing that means T:a.

Of course you need to review these changes.

2 Likes

I have always tried to write code that lets me read and write music - the challenge has been to determine “what do I want to write in order to express --musical idea–” and “how should a REPL-like process be designed that lets me build up a piece”.

So using LLMs is natural. The work of answering those questions, building a UI for myself to code in, is just as challenging. Development just goes much faster, hopefully allowing me more time to do the actual music writing (lol).

There are new challenges though! Cognitive overload! I’ve been building a little algebra of tempo maps - I can hardly remember the whole zoo and all the operations that were added! – I can go much farther down a wrong path than before!

I did enjoy little elegances and knowing how things worked (to a modest extent anyhow!) - I know much less suddenly - a little like wandering in a fog - but this can be thrilling in its own way.

We did come to these machines because we see creative possibilities in exactly “automation” after all… terrifying though this level of automation may be. The models are so much better than they were a year ago, things may become quite strange very soon.

4 Likes

I discussed such issues last week with students in the first term of our computer music study. Obviously, everyone must rethink the subject: developers, artists, teachers, and students.

IMO, the “death of coding” is only half the truth. First and foremost, it’s a welcome marketing argument for companies. As @cian points out, much real-life coding is a chore, especially when addressing clearly defined tasks. AI is a relief in this respect, and I totally understand the point of saving time and effort. James McCartney’s port of Synth-O-Matic to a modern MacOS is a breathtaking example!

But coding concrete sound and music is mostly not a clearly defined task: ideas come along the way, by experimenting, diving into the details, and discovering their musical consequences – a laborious but empowering feedback process. Cutting it off removes the edge!

On the other hand, AI will increasingly be part of artistic feedback processes. We all have to learn, experiment, and digest how this could work.

I urge students to become good at programming and smart in using AI! But also, without aesthetic reflection, all the amazing possibilities remain void.

All in all, I feel quite positive about our field from the perspective of artistic freedom. From a didactic point of view, I see new challenges: it’s increasingly difficult to convince students to spend a lot of time thinking through programming issues and practice coding. And I totally understand: why do so instead of just asking AI? Humans always tend to go the way of least resistance.

In my observation, AI and the collection of information via the internet can lead to a bifurcation of knowledge and capability. Some students can really boost their toolkit while others are overwhelmed. Not knowing what to do with a chunk of AI-generated code is just an example of such a situation. I see it so often, but there’s a way out by reflecting the causes: Recently, in our synthesis course, one student described the annoyance of such an experience. It triggered his desire to learn more about coding.

8 Likes

Thanks, you’ve expressed this more concisely than I ever could!

I want to add that this can also be true for non-musical software tasks. I’m currently (manually) reworking large parts of scsynth and supernova and while doing so, I keep getting more and more ideas. Not to mention how much I learn about the code base, particularly about corners that I’ve never looked into very deeply. All of this knowledge is important for making informed decisions. I had to shut down quite a few AI-generated/assisted PRs because they missed important context.

That being said, things like the Synth-O-Matic port are indeed a very good task for an LLM. Especially if the stakes are not very high.

1 Like

I think it would be really interesting if you described how scsynth was supposed to work and then asked AI for any ways it deviated and any bugs it could find. Then compared what you found with what it found.

“I love it [live coding], I love doing it so much. And so I thought, ‘I know what I’m going to do. I’m going to automate it so that I don’t have to do it anymore’. And this is what people tell me about once a week: You love doing this thing. Why don’t you automate it?”

I’ll develop that thought at another time, but this sums up pretty well the problem that I have with AI.

AI is great for things I don’t care about, like shell scripting or making posters (though the sameness of AI-generated posters is… well… that’s testing me).

But if I care about a task, and if AI is supposed to empower people, then I should be able to do the thing that I care about in the way that I choose. Not in the way Sam Altman chooses. Because if it has to be his way, then I’m not empowered to make the choice for myself. That’s fine (maybe?) in a job, but for my own stuff that I do on my own time… uhm. No.

I got other thoughts, but not enough time tonight.

hjh

1 Like

…sure writing the actual music is one thing (agree we should never automate this) …but writing the software that makes sound from your notation is another, no?

…I mean I use SC… which I didn’t write… to make music… why should I not use SC plus Claude-written-enhancements-that-I specified?

…and as for Altman, this is a red herring, please! - we can always use Pi GitHub - earendil-works/pi: AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI · GitHub and an open weight model if we prefer!

Do you mean the ‘sound’, or the ink/dots on screen? I don’t really think we can separate the interface/instrument/apparatus from the thing itself — I’m not to sure there is a thing… blah blah Karen Barad and whatnot.

One of the things I touched on at the end of my phd was proposing a model that highlights the value of watching people play music together (like a team sport), how this makes you feel involved in the ensemble, makes you feel like a member of the ensemble, feel every down bow, similar to how football fans feel every goal ­— I called it ensemblematic experience because I thought that sounds silly and academic… I was trying to emulate this when one of the players wasn’t human, but some computer generated being.

I think AI really makes us ask what is valuable in music… I’m sure novel sounds and processing techniques will become everyday and dull… but music is far more than sound.

2 Likes

yes agree broadly with your point. To explain though: I am using the computer more as a tool for assembling/rendering fixed pieces so for me “writing the music” is deciding what is to happen, I express this in SC code mostly these days.

I use AI to make tools to build with --for example: I’ve been bringing performance back into my work after a long time just typing… having SC extract tempo info from played snippets and then using/processing those. I used Claude to help build the tools - lets make a MonoMap class that will hold invertible monotonic maps and lets make some methods so we can capture a performance done with reference to one map and then apply another to it. There are all kinds of little edge cases (are we clamping the ends blah blah). Now can I remove jitter but leave accel/rit - and conversely? Can we quantize certain intended rhythms? We will want to implement ++ and easing. How about natural rit shapes from the literature?

All of this would be well beyond me to implement in less than a year and I’m old and greedy!

so I don’t use it for note choice, or synthDefs, or any part of what I think of as music making - ie it doesn’t assist me-the-user, just me-the-(junior)-developer if that makes any sense…

Adding my 2cts doomer rant to the pile

To me all these kind of things feel pretty pointless these days.

Why write code from hand if it someone can just generate a better version of it in a fraction of time?
Why generate code if it can be generated by someone else in the future in a fraction of the time?
Why make an effort to create some sounds or some nice sounding contraption if someone else will just come around and show you their AI generated stuff from suno or claude as a response?
Why share an idea as code if someone can just replicate the idea through a prompt or modifies the source code via a LLM?

Talking w/ some engineers these days? “Did this w/ codex, you can just drop that in your fav agent and extend it further”
Talking w/ some people who did some nice visuals at a concert? “I used claude for that”
Talking w/ some friends about some crazy stuff from gravity’s rainbow? “I asked claude about that passage and it explained to me that …”
Having a group chat w/ some friends? Someone is uploading a new group icon, generated by AI.
Talking w/ people who are writing a phd? “I just wrote a chapter of my thesis w/ chat-shi-be-dee”.
Live coders using LLMs on stage.
And on top: Walking around the city and see those awful posters everywhere.

I watched the 3b1b video and I don’t know if he or me is the one who is delusional about the upcoming changes (I also studied maths).
I felt programming was like a good excuse to spend some time on interesting problems and make somehow a living out of it, but this is now taken away. The things to discuss about is just getting narrower and narrower these days b/c people stop learning things, while everyone claiming that my work can’t be replaced b/c the LLM “is not quite there yet”, while using LLM to replace your work - and vice versa.
Everyone is just rushing to use their hyper-individualized products, but personally I was less interested in the products but in the intrinsics of a tool - what kind of new insights and approaches I could gain through this.
I think eventually all jobs which offered some “mental riddles” will disappear, combine this w/ the massive cuts in academia across the board and it provides a pretty bleak future for people who are interested in understanding stuff - I think humanity may loose its sense in things. But maybe this is just fine and I should just embrace accelerationism more and become an even worse cynic. It seems that we already lost the slot for narratives, but now we are loosing the slot for discussions.

IMO we will see a massive downshift in quality of software, as everyone starts to use their handrolled LLM stuff which just works, since professional software engineers will not be kept around much longer - in the end it will be, as always, a race to the bottom.

Feel free to change my mind on that.

I remember that we recently had a talk about the state of some documentation in sc, discussing if it is even worth the effort these days to improve them b/c most likely everybody who is trying anything new is just heading straight to AI - should we therefore write docs for AI?
If things progress as-is, I wonder if there will be people still using SC in one year from now, which seems crazy to me b/c we got the project “back on its tracks” and settled for some years w/ some beautiful new features, but why should anyone bother w/ a high level language if people stop coding?
Though maybe the “fringe” people who want to learn to craft code the old way will be attracted by sclang - I don’t think that those people would be interested in writing Java in their freetime.

I think the question regarding supercollider: should we start letting agents walk over the codebase and introduce features that would be actually great, or should we keep a more slow-paced approach as-is, where we give people a chance to develop things - but then again, who would be interested in these features if everything is just given?
B/c r/n we still follow the policy everything has to be understood by the person who submitted a patch.


I think chinese models showed that this subsidizing, at least for inference, is not necessary. You can already run them on rented hardware cheaper than subsidized subscriptions and I’d say they are quite compatible.

I think this video actually summarizes the current state of LLM coding pretty well https://www.youtube.com/watch?v=MhEL9cvLxQE

4 Likes

I agree with your points. You have raised an important distinction:

  1. using code for creating tools. You probably don’t care too much about the actual implementation as long as the tool does its job reasonably well.*)

  2. using code for sound design and musical algorithms. Here we actually want to be in control of the code and understand every detail so that we can make informed and targeted decisions. Writing the code is part of the musical process.

I see how LLMs can be very empowering, especially for people without a background in software development. At the same time, people become dependent on tools that are served by large companies. Local LLMs are nowhere near as capable as frontier models, so even when we have open weights models it’s not something that anyone can run in their homelab. Not to mention that fewer and fewer people are able to afford the necessary hardware due to skyrocketing prices. And let’s not forget that models need to be continually trained and fine-tuned, which is completely out-of-reach for ordinary people. People choose FOSS software not only because it’s free, but also because they want to be in control as users. Why are so many people surrendering to big companies? It’s not such a problem for people like us who already learned how to code. I’m more worried about the new generation who might never learn to write code by hand and always be dependent on AI companies. Maybe (local) LLMs will become a commodity, but is this something we want to bet on now?


*) Of course, tools also have an influence on what we do. They have limitations and they make certain things easier than others. This is even the case for computer music languages like Pure Data and Supercollider that seem so incredibly open ended. As soon as you spend time in different environments, you start noticing the respective advantages and disadvantages. Tools always shape the musical practice.

2 Likes