Large strings truncated by `Object.readArchive`

Questions
1

I’m having an issue with getting large strings from a file with Object.readArchive and hoping someone here can help!

I’m working on an SC interface for making dynamic interfaces with Open Stage Control, which receives JSON strings (containing widgets, parameters, etc.) via OSC to update the GUI. In my setup, I create an array (with the proper formatting, etc.) in SC and then send them over. Per now, I’m able to save the array of strings via .writeArchive and I can see in the saved file that the strings are intact. When I load them via Object.readArchive however, some of the larger strings (ie. within the array) get truncated (presumably at some character limit?) and I’m unable to load the saved GUI. The arrays of JSON strings are nested within several arrays, not sure if this is relevant or not…

Does anyone have any insight on this? It would save me a lot of headache if I could avoid breaking up the JSON strings into smaller strings, but at this point I can’t see any other solution. Any help is appreciated - thanks in advance!

2

Could you share the archive file?

3

The offending array starts at line 74 (had to add a .txt. extension to upload it here):
tooBig.txt (42.6 KB)

…which should be at Object.readArchive[3][1] if I remember correctly. Like I said, this is a save file of part of a big system, so there’s a whole bunch of other arrays that probably aren’t relevant to the issue…

4

String literals have an 8188 character limit:

("\"" ++ String.fill(8188, $a) ++ "\"").interpret.size  // 8188

("\"" ++ String.fill(8189, $a) ++ "\"").interpret.size  // 8188

Here is one workaround: JITModular/JITModPatch.sc at master · jamshark70/JITModular · GitHub

					text = text.clump(8000);
					file << "var doc = [";
					text.do { |str, i|
						if(i > 0) { file << "," };
						file << "\n\t" <<< str;
					};
					file << "\n].join;\n\n";

… where a potentially long string is split into 8000-byte strings and written into the target file as an array of strings. These are .join-ed upon reading.

There isn’t (but it isn’t that painful, in the grand scheme: clump splits it, join reassembles… it’s all there, you won’t have to do any deep plumbing).

hjh

5

Cool, this looks like the right direction! You’re right it’s not too bad, I just have to do it in a few different places (and I like whining).

Is this documented somewhere that I missed?

6

No, but I just put in a PR for that.

And… on second thought, the fact that large strings get broken in archive files is certainly a bug. So the same PR fixes that by doing the clump/join trick (but only for long strings). So, in future versions, you shouldn’t have to worry about that. (But you probably should still implement the workaround yourself, for current and past SC versions.)

hjh

1 Like
7

Thanks so much for your help and PR!