Data models for long form compositions

[wws]

Ok, I think the answer to my question is “we all have no idea what relational database table structure to use with a complicated [tonal] score”. Relatedly, the answer which does not satisfy (me anyway) is: “and you don’t want to do that anyway, you would be much happier with arrays and dicts.”

After I finish bombing my aural skills test tomorrow I will write up a sketch of how I have since thought about a score/ motif database, and how I would use it in a rogue-like music exploration game with Super Collider running the ascii interface and rendering the music.

Thank you to everyone interested enough in keeping the thread alive!

[wws]

Here is the idea for the game:

It has multiple levels, each a grid of squares. Each time you move onto a new square a musical motive sounds (2-5 notes usually). A path is formed by a series of squares where each point is a single step from another square. Once you trace the correct path for the level (a song, say Happy Birthday), it opens the next level for you to explore. Some squares will just be dissonant crap, some might be close to the right motive but not quite, whatever you like when you design the level. Imagine something like Dungeon Crawl Stone Soup, or Rogue, or the like.

So the trick is to elegantly store all those motives in their correct (X, Y, Z) coordinate, the paths traced by the player, the current position and other state, and probably a whole lot else (I am NOT doing that with just a bunch of arrays and dicts, thank you very much…)

Architecture: SuperCollider is a client that handles input from user, rendering the map (just ascii), and rendering the motives in sounds. There is a SQL database server that uses stored procedures called by SC and sends back easy to parse stuff for the music and simple ascii art for the level maps. Besides leveraging SQL for handling lots of data, I think the database server architecture could be extended to multiplayer somewhat naturally, though I don’t know what would be the point.

I am still thinking about the data model, will post as I have more time and better developed ideas.

Anyway, would be interested in any thoughts anyone has.

[smoge]

Hello @wws

Some ideas…

• Instead of predefined melodies, why not infinity series or a similar
  quasi-repetitive patterns?

• Streams can interfere with each other, and modified by the player and
  environment

• Some goals that can happen in particular positions and situations. For
  example, streams that cancel out each other; or when sounding together, a
  special melody is heard through the interaction.

As an example, let’s suppose a pattern similar to Nørgård’s “infinity sequence”.
It can be a good start because is well-balanced between repetition and
variation:

• The Danish composer Per Nørgård’s “infinity sequence”, in an attempt to
  balance repetition and variation.

• Equation: $a(2n) = -a(n), a(2n+1) = a(n) + 1, a(0)=0$

• My implementation, from years into the past: https://github.com/smoge/InfinitySeries

• Relevant blocks for us:

Pinf : Pattern {
	var <>start=0, <>step=1, <>length=inf;
	*new { arg start = 0, step = 1, length=inf;
		^super.newCopyArgs(start, step, length)
	}
	storeArgs { ^[start,step,length] }
	embedInStream { arg inval;
		var outval, counter = 0;
		var cur = start.value(inval);
		var len = length.value(inval);
		var stepStr = step.asStream, stepVal;
		while { counter < len } {
			stepVal = stepStr.next(inval);
			if(stepVal.isNil) { ^inval };
			outval = cur;
			cur = cur + stepVal;
			counter = counter + 1;
			inval = outval.infiniteNumber.yield;
		};
		^inval;
	}
}
+Number {
	infiniteNumber {
		^this.asBinaryDigits.filterLeftZeros.infinityDigits
	}
}

Borrowing ideas from the pattern above to use in the Game

• added porosity so vectors can operate

// Pervasive to data from the game (positions, levels, etc)

// It's raw, ugly, quick, dirty, but it's here mostly to show something
// already working, so one can get started just having fun hacking this
(
~infinitySequence = { |start=0, step=1, n|
    var seq = Array.newClear(n);
    
    seq[0] = start;
    (1..n-1).do { |i| seq[i] = if(i.even) { seq[i/2].neg } { seq[(i-1)/2] + step }};
    seq
};
~getNote = { |x, y, z, start, step, gridWidth, gridArea|
    var position, sequence;
    
    position = x + (y * gridWidth) + (z * gridArea);
    sequence = ~infinitySequence.(start, step, position + 1);
    sequence[position]
};
~makeGamePattern = {
    var durBase = rrand(0.05, 0.3);
    var durMult = rrand(0.9, 1.1);
    var detune = {rrand(1, 5)};
    var octaves = {[rrand(3, 6), rrand(2, 6)]};  
    Pbind(
        \degree, Pfunc({ 
            ~getNote.(
                rrand(0, 9), rrand(0, 9), rrand(0, 9), 
                0, 1, ~gridWidth, ~gridArea)}),
        \dur, Pfunc({ durBase * durMult }) + Pwhite(-0.025, 0.025),
        \detune, Pfunc(detune),
        \octave, Pfunc(octaves))
};
// Some sounds:
~gridWidth = rrand(1, 200);
~gridArea = rrand(1, 200);
~currentPattern = ~makeGamePattern.value.play;
)

// stop () cross-fading, or other merging strategies would work fine here)
~currentPattern.stop;

// new variant with different input data
~gridWidth = rrand(1, 200);
~gridArea = rrand(1, 200);
~currentPattern = ~makeGamePattern.value.play;

Some ideas

• Vectors could do things like these, trying to keep simple here

1. Morphing between different patterns based on game progress.
2. Poly-rhythms can happen when the player’s position in borders or something like that
3. Increasing rhythmic complexity as the game progresses, example: Ppatmod
4. Interrelated patterns for multi-dimensional movement.
5. Harmonic regions (seems you’re working with tonal material), in this case,
   change harmony is something powerful to use

[grahamsw]

The answer is, it doesn’t matter at all what data store you use. I don’t think relational databases are a good fit, but if you like them then go for it.

For my own uses I just don’t see the amount of data getting to the scale where it won’t all fit easily in memory.

What I’m using is: an object definition that fits my needs

(
    \name: "Ex 7: The Rhine",
    \tempo: 80/60,
    \key: \e_flat_major,
    \transpose: 3,
    \time_signature: [6, 8],
    \parts: [
            (
                 \midinote: [63, 58, 55, 58, 63, 65, 67, 63, 58, 63, 67, 68, 70, 67, 63, 67, 70, 75, 79, 75, 70, 67, 63, 58],
                 \dur: [0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25, 0.25]
            ),
            (
                \midinote: [[55, 63], [58, 65], [63, 67], [65, 68], [67, 70], [70, 75], [75, 79]],
                \dur:[1.25, 0.25, 1.25, 0.25, 1.25, 0.25, 1.5]
            )
    ]
);

each motive goes into its own file, named <something>_motive.scd, in a folder.

Then I call this to load all the files into an array of objects (events)

PathName( foldername).files.select(~is_motive).collect({
    |f|
    f.fullPath.load
    });
 };

I can add motives, without changing anything else, I can filter them on key/time signature, tempo, instantly (there are only ever going to be a couple of hundred of these).

I’m going to have to add some more details to the objects, quant info for one, but it’s just a bunch of files.

But if you’re comfortable with mySql, say, then go for it. It honestly doesn’t matter. You’re going to need a way to enter and edit the motives, and a way to get them by feature. Whatever you use for that you’re going to end up with the same thing - an object with the information you need to do the thing you want.

My software development advice is: build interfaces. then build things that implement the interfaces. Then make the implementations more complicated only if you need to.

You want a system that allows you to query for (what I’m calling) motive information, and once you get that information you’re going to play it somehow.

Once you decide on the information, build something super simple that returns motive info - like something that just gives you a random motive, no matter what you ask for.

Then build a simple version of the “play motive” system you want (ASCII art and all). Then elaborate each of these systems as necessary.

The advantage of this is that you have two, independent, systems to work on, rather than one big complicated one. This makes it easier to collaborate with others (including your future/past self), and reduces the amount of information you have to retain in your brain while working.

Edit: the main advantage of this is that you get something working quickly, and you get to hear it, and see if it’s worth pursuing. And hopefully you get to spend your time on the bits that matter most

[smoge]

@grahamsw

My software development advice is: build interfaces. then build things that implement the interfaces. Then make the implementations more complicated only if you need to.

That’s quite interesting and makes sense. I’ve also heard people saying “Write the tests first, than the code” )))

[grahamsw]

@wws

So the trick is to elegantly store all those motives in their correct (X, Y, Z) coordinate, the paths traced by the player, the current position and other state, and probably a whole lot else (I am NOT doing that with just a bunch of arrays and dicts, thank you very much…)

Oh, yes you are!

You want schemaless storage, because you’re going to be modifying the data structures ALL THE TIME during development.

If you can’t store everything you need in JSON then you can’t store it in a database. The functional equivalent of JSON in SuperCollider is the event (well, the dictionary, but in practice the event)

Once you’ve got your JSON/events stable, then sure, if it makes sense to use a DB then go for it. Development will be much faster if you do that as a last step.

At least it would be for me. YMMV