Data models for long form compositions

Questions
1

Hi there,

Does anyone know of books or whatever describing, for lack of a better term, data models for long compositions? Like, how would one save a 16 minute fugue in a SQL database?

Even better, how can one store it and maintain some structure, so for example store a single motive but also some transformations that on that motive that allow it to be developed? (Rather than just a big table of notes and timings.)

This is a pretty big question, I appreciate any musings or links as well as direct answers.

There is a video game angle to this, but I haven’t quite formulated this in my mind.

Thanks!

2

I regularly use a kind of text-based notation called “Panola” which serves me well as the starting point for all kinds of transformations and concatenations.

You can see a video of what I mean here:

The added value of an “SQL database” in your question is less clear to me. What kind of queries would you like to perform? Depending on what you hope to get out of the database, you’d probably need to carefully design your table structure.

2 Likes
4

The added value is mostly that I am curious how it would be done.

If nothing else, you could query “what are the first violins playing on mm 43 to 47,” but I think if you had a more structural schema you might be able to analyze and modify thematic elements, like “what is motive #2 in retrogade.”

6

I don’t disagree with Nathan. System Building Syndrome is real. You just have to be self-aware and catch yourself when it takes over. Everyone on this mailing lists is attracted to it’s dark power, otherwise we would be using a different language, haha.

However, I like systems. In improvised music, if you have a good system, you can make music with it for years without getting bored. Even when I am composing, I need a system before I can compose. Before I find an algorithm, I can’t write the music. After, the music just writes itself.

Sam

2 Likes
8

@nathan, just curious how would you compose a 100 measure, four part piece? What infrastructure is there in SuperCollider to organize that many notes, in a static predetermined composition?

I have to say, typing in Supercollider code for (let’s just say) 100 x 4 x 6 (measures times voices times average notes per measure per voice in 4 4) is 2400 things. I would rather use Dorico or Sibelius or Musescore and enter normal music notation, because it just sounds awful to enter all that stuff in a programming language. And that’s not a particular big composition, really.

All this aside, besides “System Building Syndrome” I think there is also “I will answer a different question syndrome”… No offense, I just really want to know how to store a long form classical piece in a SQL database in an elegant fashion, that’s why I asked that question.

9

MuseScore is an open-source notation package, meaning you have access (in its source code) to some concrete answers about data structures for western music. (Or lilypond, for that matter – which actually I prefer when I need to produce notation.)

Edit: I don’t mean this facetiously, btw – just observing that the SC community’s level of grappling with this technical problem is likely to be a lot less profound than that of a software community focusing on that specific problem – so, go where the water is.

hjh

10

if you’ve somehow stored motive #2 in your schema you can calculate its retrograde on the fly - you wouldn’t necessarily have to store that as well

i do think you would end up with different schemas based on whether the intent is performance or analysis - same as in any software project where you probably have one schema for realtime transactions and one for reporting in your data warehouse

if you kinda try to reverse engineer a daw you would probably end up with a many-to-many relationship between tracks and timeline entities

since this is a supercollider forum - what more do you need than dictionaries and arrays - it seems those would be enough to start modeling some structures and maybe clarify some concepts

11

But why do you think you need an SQL database? Sounds pretty overkill to me. I agree with @droptableuser: why not just arrays and dictionaries?

12

Arrays, Dictionaries and saving / loading as archive. Only reason I’d see going a SQL route is if your are planning complext relationships - but even then, dictionaries with keys (and saving references to keys in your objects) should do it just fine.

1 Like
13

Because even for a little effort, you get an immense payoff in how you can fuss with your data, extend your model, and undertsand the data independent of the code.

For ex: Even if you are just storing a whole lot of notes, you can do that in a table with columns <motive #, timing, voice #, note>. Yes, you can do this in an array of dicts (or vice versa), but (1) it’s tricky to ask arbitrary queries like “give me the 2nd voice of the 3 and fourth motives ordered in reverse timing” and (2) it’s even trickier to extend like adding a table for each voice with timbre setting etc. ((1) is a simple query if you know what you are doing)

Anyway, whatever floats your boat is fine. I hate handling complex data in arrays and dicts because I am used to SQL and it is so much more powerful

14

Here, SQL lets you combine them into one query (and it’s the database’s job to figure out the best way to execute it) – indeed, SC can’t do that. SC could do the query, but you’d have to write the execution plan into the code, splitting it up into multiple .select and .collect operations.

Ultimately it’s up to you how you want to work – just because someone here says SQL is overkill, doesn’t mean that your goals and preferences align with theirs.

Adding items to dictionaries is pretty easy in SC, though.

But that part of the discussion is about how-to-do, where the data structure design question is about what-to-do. I have very little to say about that, because my SC work isn’t along those lines, and if I need something like a score, I’ll use lilypond instead of SC. I expect a lot of SC users feel similarly. But, MuseScore or lilypond devs would have a lot to say about data structure design, and have probably already handled cases you haven’t considered yet. So it’s a valid approach to your question, to raise it in those software packages’ support communities.

hjh

15

Trying to store music notation in an SQL database gets extremely messy very soon (depending on what kind of queries you hope to make) because there are so many possible relations between music lines (horizontal and vertical, intra-staff, and inter-staff, harmonies/melodies/rhythms/…).

What you are looking to do probably is a more natural fit for a system like music21 (music21 Documentation — music21 Documentation) and perhaps also scamp (https://scampsters.marcevanstein.com/)

These systems are written in python, not supercollider, but you can fairly easily drive supercollider (for synthesis e.g.) from the python code using OSC communication.

16

Hello @wws

Some ideas to get started:

In music, we can think of “points” as specific events, like a note at a particular time, and “vectors” as transformations or movements between those points, such as intervals or rhythmic shifts. For example, a point in pitch is a specific note, while a vector represents the interval between two notes.

Similarly, a point marks a particular moment, and a vector represents a duration or rhythmic displacement. This approach abstracts musical transformations, like transposition, making it easier to define, analyze, and compose by applying the same changes across different elements. Transformations like inversion or augmentation act as vectors that shape the structure and relationships between voices.

To work nicely in tonal contexts (if that’s your thing), you need a framework that accounts for diatonic and chromatic intervals. Music21 and music-suite do this; you can use a simplified version of this idea. I posted minimal code examples in this forum a while ago, enabling accurate interpretation of intervals, key signatures, and accidentals depending on your composition/analysis goals. (Although I don’t work with tonal materials, this aspect works well when dealing with intervals and accidentals in general, which can also be helpful).

This fits a hierarchical structure to represent elements like themes, voices, and transformations, making it cool for certain types of music. A type system (e.g., Music, Score, Voice, Note, Rest, etc.) can capture musical and temporal details. At the same time, functions for transformations (such as transposition, inversion, and augmentation) allow for modifying motives and other elements. There are several examples around.

While hierarchical structures work well for representing themes, voices, and transformations, another interesting approach is using prefix trees.

Prefix trees (which is also ‘hierarchical’ but different :stuck_out_tongue_closed_eyes: )allow for musically meaningful operations like combining and transforming patterns across different “slices” of a score. Unlike hierarchical systems, prefix trees efficiently encode repeated patterns by sharing common prefixes, reducing redundancy. They are beneficial for representing recurring musical motives or transformations across multiple dimensions (e.g., vertically in harmony, horizontally in time, and other “slices”). By traversing the tree structure, you can quickly identify variations or transformations within a score and manipulate these across distinct, contextually musically meaningful slices.

PS. There is no problem writing some scripts in python or another language, converting data from sc (for example, Dictionaries) into JSON, and vice-versa back to SC. You don’t need to recreate an extensive system in sc if you don’t have a good reason.

1 Like
17

First off, I love this question: what is a good data model for long compositions, and how do you manipulate them?

This kind of modular “video game music” like approach seems like something that Supercollider should be really good at, and there are some things you do with Pchain, Pbindef etc, but this is something where everyone seems to be inventing their own solution.

There’s no getting away from the complexity of music - there’s a large number of notes in a 16 minute fugue - but typing them into a Pattern is going to drive anyone insane. (Scores are the way they are for a reason.)

One of the things that fascinates me is the range of timescales involved in music, and one of the difficulties with electronic music is that the same person is responsible for all of these timescales, from “conducting” level decisions that play out over minutes, down to the sample level changes. In non-electronic music there are conductors, composers, musicians, and instrument makers, all contributing at different temporal scales. (I think that this is why so much electronic music is kind of boring.)

Designing the appropriate data structures, the ways of manipulating and storing them is a very, very interesting question. I’d love to talk about this more.

3 Likes
18

Hello @wws

Two points:

The SC Way

SuperCollider is the undomesticated child of programming. People who use it are often into some pretty out-there stuff. They’re not usually trying to recreate traditional sheet music or CS cutting-edge concepts - they’re hacking stuff to create cool art.

When someone starts a project in SC, they’re not typically thinking, “How can I make this work for every possible music theory concept?” Nah, they’re more like, “How can I make this weird sound I’m imagining?”

It’s pretty different from something like LilyPond. LilyPond is more of a team long-term project, trying to cover all the bases and adding new features when people need them. They have no problem with sponsoring particular features that cover rare cases. It’s normal for them.

SC, not so much. it’s more of a “figure it out yourself” vibe

But here’s the cool part about the SC crowd - they’re super into sharing. People are always throwing code snippets and ideas around. It’s less about making everything the same and more about inspiring each other to try new things.

Databases in Music

Now, let’s discuss databases. This might be a good idea. Do some research there!

See Neo4j. It’s this graph database that, from what I can tell without being an expert, could be pretty cool for music theory, notation, composition, and analysis. Why? Well, music is messy, right? Everything’s connected in weird ways. Neo4j (a graph database) handles that kind of tangled-up data well.

Imagine seeing all the connections in a piece of music before you. From what people told me and what I could grasp, Neo4j could do that. (Again, I’m not an expert; I am writing from unreliable things my brain remembers from what II read and heard)
Cheers

19

+1 for Neo4j here - no idea if it actually makes sense, but I’ve been looking at it for work related things, and it’s amazing.

Currently I’m trying to get my head around MusicXML. My plan is: use MuseScore or something to edit score, export to XML, read to Panola like stream for SC.

I know, SC isn’t really aimed at producing “traditional sheet music”, but goddam it, it should be possibe, and easy.

1 Like
20

It’s possible, no doubt. It’s just that it’s not the way people write stuff in SC. SC users focus on something more specific to the kind of music they do, not some general implementation, as one can easily find in other languages. I’m not blaming SC users; it’s just that the other languages (python, c++, javascript) are beyond the “threshold of immortality” and have much more resources and people writing stuff for general use.

21

I’ve written a lot of sheet music using SC, so it is definitely possible.

In this piece, for instance, section IV, the double duo (around 11:30-14 min) superimposes two algorithmically generated (in SuperCollider) scores - one for two pianos and one for two percussion. I exported these to midi with SimpleMIDIFile and loaded them into Sibelius to fix up.

http://sampluta.com/compositionSevenSystems.html

I wouldn’t want to write an entire piece this way, but parts for sure.

Sam

22

As an alternative, this might provide a starting point.

Even though it mentions musescore 3.x in the description, I’m pretty sure I’ve used it with musescore 4.x as well at some point (although it’s been a while).

It’s far from perfect (limited in what information it captures) and might not meet your needs at all. If I recall correctly I hit a lot of random limitations related to what score data a musescore plugin can access. Musescore plugins unfortunately seem to have been added as an afterthought instead of being an integral part of the design from day one, so in the end going the musicxml route may turn out to be the better option.

23

I’ve just taken a look at MuseScore and Music21 (the python library)

Turns out its pretty simple to write a score in MuseScore - either with a midi keyboard or just on the laptop, export it as MusicXML, read that into Music21, and get that to spit out the parts as a lists of midi notes and durations that you can use directly in a pattern

I think this is everything I need for this right now. You need to be a little artificial in that you can have a separate part for any rhythmically distinct part - each line has to share the same rhythm. But at least you can have chords on the same stave, so serves (me) as a more convenient way of putting themes and motifs into SC patterns…

Easy too (like 20 lines of Python)

1 Like