Data Bending / Open Raw in SC

Hi,

Many will know that data bending in Audacity is a very easy process (for example, importing an image file as raw data and playing it).

I’m trying to figure out if it would be possible to do the same in SC. Is it doable?

Thank you in advance.

Warning: noob here with SC , take my advice with a mountain of salt

I would guess that using a Buffer and reading the image file and converting the rbga values per pixel, assuming you read from a PNG file for example , to sample data and then playing back the buffer should do the trick.

You could also use the Image class from the gui to read from an image file , then you can use the .getPixel(x, y) method to get each pixel as RGBA integer . You can also use the Image class as a GUI element to allow you to draw on top of the image and change the sound real time. Image has a draw method but you can also use setPixel method to directly change each pixel.

Of course the real deal here is how you interpret the image file which will define the musicality of the sound in question. So dont expect to get the same sound results as you may get in other software, a synth can interpret an image file any way it wants to. Unlike a audio file an image file may not represent a sound as you may expect.

1 Like

Thank you for your answer @kilon. Noob here as well.

However, I was more interested in an automatic ‘interpretation’ of the file. Something like:

f = SoundFile.new;
f.openRead("/my/path/whatever.pdf"); //or .jpg, .mov, etc.
f.play;

Could SC interpret the raw data somehow? Of course this doesn’t work, but maybe there’s a way to do it. Or maybe not…

SoundFile assumes that there will be an audio file header, which of course PDF doesn’t have.

But there is File, which makes no such assumption.

Files simply contain a stream of bytes. It’s up to you how you want to use the bytes. To turn arbitrary data into (noisy) audio, a pretty safe bet is to read integers – getInt8, getInt16, getInt32 (or reverse the byte order with the little-endian variants: getInt16LE, getInt32LE). Reading floats would be a bad idea because it’s very easy to have NaN or infinities or denormal values accidentally (you don’t want those). Integers at least can’t be broken.

There’s no built in getInt24 but that would be pretty simple to write as a function: ~getInt24 = { |file| (file.getInt16 << 8) | file.getInt8 }.

Then floating-point signal values would be:

  • 8-bit: y / 256
  • 16-bit: y / 32768
  • 24-bit: y / 8388608 (2 ** 23)
  • 32-bit: I wouldn’t bother with this, but: max(y, -2147483647) / 2147483647.

Then loadCollection to a buffer.

Multichannel buffers will treat the channels as interleaved, which can be a cool effect.

I saw a piece by Yota Morimoto once where he used executables as an audio source – some surprisingly beautiful sounds to be found there.

hjh

3 Likes

Thank you @jamshark70. I will look into that. :+1:t3: