[Tutorial] Coding a 2D Corpus Explorer

Greetings,

Here’s a five-video tutorial building a 2D corpus explorer using FluCoMa. It goes through:

  1. Slicing audio using FluCoMa slicer objects
  2. Analysing audio slices in non-real time for spectral centroid and sorting slices according to spectral centroid
  3. Plotting slices in 2D space using spectral centroid & loudness
  4. Import a whole folder of audio files for slicing, analysis, & plotting
  5. Using the dimensionality reduction algorithm UMAP to reduce 13 MFCCs coefficients to 2 dimensions to plot

Let me know what questions you have!

Cheers,

Ted

17 Likes

Thank you for this! I just downloaded the code at the end of part 4 (plotter-4.scd) where you wrapped all of the previous work into one big function. When trying to run this function, the server gets stuck at some point in the analysis:

~twoD_instrument.(FluidFilesPath()); // run the function

-> a Routine
bufnum: 0
numFrames: 22268421
numChannels: 2
sampleRate: 48000.0

done
done
average seconds per slice: 0.31325147704254
slice 0 / 1481
slice 1 / 1481
slice 2 / 1481
slice 3 / 1481
...
slice 296 / 1481
slice 297 / 1481
slice 298 / 1481
slice 299 / 1481 -> stuck! no further console output

This is on a fresh build from source of both FluCoMa and SuperCollider on Arch linux, without any other Quarks/Extensions installed.
Changing if((i%100) == 99){s.sync}; to just s.sync; works and the analysis runs to the end just fine - I know you discussed this in the video (not syncing often enough), but unlike you, I don’t get any error messages about the FIFO clogging up (exception in real time: alloc failed ...), it just gets stuck without throwing any errors.

1 Like

you are such a great teacher, thanks for making these videos :slight_smile:

1 Like

Ah, interesting. Thanks for reporting. Unfortunately, I don’t have a Linux machine to test on, but I’m glad you found an edit to the code to continue working with it!

If you’d like, you can open a bug report on our GitHub repo and see if someone there can run a some checks perhaps and see what might be causing it! If you’d rather not delve into there, I’d be happy to open the issue on the repo as well.

Cheers,

T

1 Like

Thanks for the videos. I’m about to start with #3. As I know nothing of all what is presented so far I was a bit confused, diving straight in. I felt a bit lost, where is this going? The very first part of #2 clarified a bit.

For me a #0 video that introduces the concepts, ideas and goals a bit would be very welcome.

So, off to #3 now.
Cheers.

1 Like

I’ll see if I have time today to open up an issue. Thanks again for the videos! I thought they were well paced and easy to follow along with otherwise. Already had some fun “sorting” some old Grandmaster Flash tunes sort of like the old Automatics Group edits, FluCoMa makes this a breeze.

1 Like

Thank you very much Ted !
Your tutorials are very well explained with precise examples of how to use the library !
Flucoma really allows to open this new world of concatenative synthesis and neural network control to SuperCollider ! It is a very important contribution !!

Otherwise, I just tested the examples on an Apple M1 (I have compiled both SC and Flucoma in native M1) and I have problems with buffer handling. It seems to me that there is an OSC overload problem that causes the server stuck, like what bovil43810 describes. After a long time of waiting it works… I don’t have another M1 at hand to test but it seems to me a problem in SuperCollider server on M1. I get the same thing when I load many buffers at once, on an Intel they load fast, on the M1 it takes a lots of time.

Is there anyone who can test the examples on an Apple M1 natively ?

All the best,

José

1 Like

Hi @Jose, thanks for the encouragement! I’m sorry to say, I don’t have an M1 at my disposal to test for you. Could you raise this issue for us at the GitHub repo and see if one of my colleagues could test for you!? It’ll be important for us to find out if there’s something weird going on with FluCoMa on scsynth!

Cheers,

Ted

Hi @bovil43810!
Hi everyone!

When trying to run this function, the server gets stuck at some point in the analysis
Changing if((i%100) == 99){s.sync}; to just s.sync; works and the analysis runs to the end just fine

Please @bovil43810, did you solve the module issue using ‘if((i%100) == 99){s.sync};’?
I am also studying the video tutorials from FluCoMa but on Ubuntu Linux…
and I also got stuck on this…

Thank you!

Hi @Ivan_Simurra,

Let me know if this thread is helpful!?

Best,

Ted

1 Like

Hi @tedmoore!
Thank you for the feedback!

Reducing the group size actually seems more efficient.
At least it worked here with Ubuntu
I didn’t see an increase in processing time using the following:

var every = 14; // analysis stops working when (every >= 15) if((i%every) == (every-1)) { s.sync };

I still haven’t tried the other alternative, with bundle, but for now it worked!

Thank you very much
Keep rocking guys!

1 Like

@Ivan_Simurra, I’m glad it’s working! Let me know if you have any other questions about the code!

T

1 Like

Hi! These tutorials were crazy well done, and I got to try this whole system out on my project (a pressure matrix sensor with 195 sensor points on Arduino Mega, run from supercollider).
I was curious about examples of controlling FluidPlotter pointer function with something else than a mouse, as my current code uses the above mentioned matrix to enlighten squares on a GUI that I’m overlapping to the FluidPlotter view. I mainly have two questions:

  • is there a way to interact with FluidPlotter without a mouse, for example by specifying X and Y coordinates of the pointer via an external controller and using something else than mouseMoveAction? Or perhaps is it more convenient to leave FluidPlotter alone and work with FluidKDTree in some other process to update my custom GUI?
  • is it possible to highlight more than one point at a time on FluidPlotter, for example having a tactile interface tracking more than one position simultaneously? That would be really cool to move through the sample space with more than one finger :stuck_out_tongue:
    Hope this makes sense :smiley:
2 Likes

Whatever x,y position you want to find the nearest neighbor of just needs to be in a two sample buffer and then pass that buffer to the KDTree’s kNearest method. That x,y position could come from the Plotter or could come from anywhere!

Yep, totally possible.

You can pass an array of identifiers to the plotter’s highlight_ method and it will highlight all of the IDs in the array.

1 Like

Nice!! Thanks for the prompt answer :smiley: Makes total sense, will post online the results asap

2 Likes

I’m currently in the process of discovering a whole lot of funny things! I am posting the updates in my code in a dedicated github repo (WIP) but essentially it works.

What I’m doing (to my understanding lol) is surely not yet efficient nor fully functional, but it manages to play slices passing the pressure matrix coordinates pair buffer to kNearest. I’ll prolly come up with questions in some other thread :slight_smile: very fun tho

2 Likes