Possible to break huge sysex into smaller pieces?

Questions
1

If I send a sysex msg to an external hardware device using the MIDIOut.sysex method, it works correctly for small sysex messages, but as soon as they grow too big, it fails.

I suspect it may have to do with either some internal buffer size being too big or not sufficient delay between sending successive buffers, causing the hardware synth to choke.

I was wondering now if I could somehow break up a larger sysex message in smaller chunks myself, and send them to the synth one way or another from sclang? Is there some lower level api one can use for this purpose? Or any alternatives?

2

A sysex message, in SC, is just an Int8Array. Within SC, of course you can do anything you want to the array.

// Split 20 values into 5-member clumps
Int8Array.fill(20, { 128.rand }).clump(5).collect(_.as(Int8Array))

The problem is, SC can’t guarantee that the receiving device will understand the message, after you break it up. We don’t have a lot of control over that.

“It fails” is a bit lacking in troubleshooting detail. Is it an error? Something locks up? Or, does it seem fine in SC but the external device doesn’t respond?

Any guess as to the threshold where it breaks down?

I poked around the source code a little bit, but I can’t see right away where there’s a hard limit on the message size.

Mac, Linux or Windows?

SC version?

hjh

3

Sorry I was a bit vague on details.

  • Latest supercollider built from git on arch linux 64bit system (but the behaviour’s been like that since “forever”)
  • The sysex itself works perfectly when sent using MidiOX on windows, but the same sysex downloaded from supercollider in linux doesn’t work, i.e. the synth doesn’t seem to react to it. Smaller hand-crafted sysex (e.g. to upload an alternative tuning table) work just fine in linux.
  • The sysex that causes trouble contains an OS update, and it’s over one MB long
  • Normally long sysex are automatically split into smaller buffers and sent one by one, but some devices need sufficient delay between receiving successive buffers to allow processing the information. MidiOx has some options to set up these buffer sizes and delays, but supercollider has none of these as far as I can tell.
4

Hm, if it works with MidiOX, then I would use something (maybe even SC!) to capture the messages that MidiOX is sending. Then you could try to reproduce those in SC.

MidiOx has some options to set up these buffer sizes and delays, but supercollider has none of these as far as I can tell.

Buffer sizes: See my ‘clump’ example, above.

Delays: Routine or Task.

We have the mechanisms to handle both of those requirements. It’s just that MIDIOut does not automatically do it for you.

What I don’t know is if MidiOX is doing something special to the message format to tell the receiving device that there’s more sysex coming. It might be as simple as sending a packet, and the sysex isn’t over until it receives the terminating status byte. But I haven’t done this myself, so I’m only speculating. Capturing the data from MidiOX would give you a sure answer to those questions.

EDIT: Unless MIDIOut automatically appends the “end-sysex” byte. In that case, manually splitting it up wouldn’t work.

hjh

5

Yes, that was my concern since the documentation is so explicit about mentioning that “on windows platforms” the message has to contain a complete sysex message (which sounds like on other platforms it is not needed and hence stuff is added automatically). I’ll have to conduct some experiments. Thanks for the replies.