How to send audio data to SuperCollider via OSC

Questions
1

Hi everyone,

I am trying to send audio data from JavaScript to SuperCollider via OSC. Since OSC does not support raw Blob types directly, we believe it must be sent as a Uint8Array.

I tried converting the Blob to a Uint8Array before sending it via OSC, but SuperCollider is only receiving small Int8Array data. I suspect there is a limit on the amount of data that can be sent. Is this the correct format for sending binary data to SuperCollider over OSC? Do you know a better alternative?

There also seems to be a limitation on the number of messages. In this example, we can only send 152:

(
~totalChunks = nil;
~receivedChunks = Dictionary.new;

OSCdef(\test, { |msg|
    var index = msg[1];
    var chunk = msg[2];

    if (index == -1) {
        ~totalChunks = chunk;
        ("Total expected chunks: " + ~totalChunks).postln;
    } {
        ~receivedChunks[index] = chunk;
        ("Chunk received:" + index ++ ", Total received:" + ~receivedChunks.size).postln;
    };

    if (~totalChunks != nil and: (~receivedChunks.size == ~totalChunks)) {
        "All chunks received!".postln;
    }
}, '/test');
)

(
~sc = NetAddr("127.0.0.1", 57120);

~int8Array = Int8Array.fill(44100 * 8, { 127.rand2 });
~numChunks = (~int8Array.size / 1024).ceil;

~sc.sendMsg("/test", -1, ~numChunks);

~int8Array.clump(1024).do { |chunk, i|
   ~sc.sendMsg("/test", i, chunk);
};
)

I would appreciate any suggestions or experiences regarding sending audio data to SuperCollider. Thank you!

2

have you looked at this:

3

@Sam_Pluta AOO is for real-time audio streaming, but I don’t think that’s what @retroriff is trying to do.

@retroriff

Since OSC does not support raw Blob types directly, we believe it must be sent as a Uint8Array.

I’m not sure what you mean here, OSC blobs are literally raw bytes. Which JS OSC library are you using? What does the documentation say about OSC blobs?

There are few issues in the code.

#1 You need to make sure that ~numChunks is an Integer:

~numChunks = (~int8Array.size / 1024).ceil.asInteger;

#2 Unfortunately, .clump does not seem to preserve the original type, instead it always returns an Array! In your case this has unexpected and drastic consequences: NetAddr.sendMsg treats Array as a nested sclang-style OSC message, converts it to raw bytes and sends it as a blob. (This behavior is not actually documented, but I think this is meant for OSC completion messages, such as in SynthDef.add or Buffer.write.) This means your audio data is accidentally interpreted as an OSC message! You need to make sure that you are really sending an Int8Array.

Here’s a working version of the sending part:

(
~sc = NetAddr.localAddr;

~int8Array = Int8Array.fill(44100 * 8, { 127.rand2 });
~numChunks = (~int8Array.size / 1024).ceil.asInteger;

~sc.sendMsg("/test", -1, ~numChunks);

~numChunks.do { |i|
	var index = i * 1024;
	~sc.sendMsg("/test", i, ~int8Array[index..(index + 1024 - 1)]);
};
)

Regarding the lost messages: every UDP socket has a send and receive buffer with a fixed capacity and if you send messages at a faster rate than the receiving end is able to process, packets that do not fit into the receive buffer are simply discarded. That is one reason why UDP is considered an unreliable protocol.

The default capacity of the buffer depends on the OS and its settings. (Applications can also manually set the socket buffer size, see GitHub · Where software is built) Windows, for example, has a notoriously small default size, typically 65 kB. In your case this corresponds to about 64 chunks of 1024 samples. And indeed, here on Windows 10 I can only send about 64 chunks.

This means you must be careful not to send too many UDP messages in one go. Instead, you should wait for a few milliseconds between every N messages to avoid overflowing the receiver’s socket buffer. See this recent post for how to do this in sclang: How best to slow down / queue outgoing OSC messages - #2 by Spacechild1

In JS you can do something very similar, but you would need to implement your own wait function: What is the JavaScript version of sleep()? - Stack Overflow.

Side note: why is ~receivedChunks a Dictionary and not an Array?

4

@Spacechild1 I tested your code and it works as expected. Here is the final version using an Array instead of a Dictionary:

(
~totalChunks = nil;
~receivedChunks = Array.newClear(~totalChunks);

OSCdef(\test, { |msg|
  var index = msg[1];
  var chunk = msg[2];
  
  if (index == -1) {
    ~totalChunks = chunk;
    ~receivedChunks = Array.newClear(~totalChunks);
    ("Total expected chunks: " + ~totalChunks).postln;
  } {
    ~receivedChunks[index] = chunk;
    ("Chunk received:" + index ++ ", Total received:" + ~receivedChunks.reject(_.isNil).size).postln;
  };
  
  if (~totalChunks != nil and: (~receivedChunks.reject(_.isNil).size == ~totalChunks)) {
    "All chunks received!".postln;
  }
}, '/test');
)

(
~sc = NetAddr("127.0.0.1", 57120);

~int8Array = Int8Array.fill(44100 * 8, { 127.rand2 });
~numChunks = (~int8Array.size / 1024).ceil.asInteger;

~sc.sendMsg("/test", -1, ~numChunks);

~numChunks.do { |i|
  var index = i * 1024;
  ~sc.sendMsg("/test", i, ~int8Array[index..(index + 1024 - 1)]);
};
)

The library I am using is osc-js, and this is what I found in their documentation:

blob new Uint8Array([11, 44, 31]) for any kind of binary data

That’s why I started testing how to send and receive Int8Array in SuperCollider. Thanks to your help, I finally got it working.

Now, the next tricky part is converting the received Int8Array into a Buffer. I haven’t found any examples, but I think the first step should be converting it into a FloatArray.

Thank you all for the help.

1 Like