How does sclang REPL work exactly and what is -i?

Questions
1

Hello,

I currently write a supercollider kernel for the Jupyter Environment but struggle with sclangs REPL, especially multiline expressions.

The question is how to execute sclang code in the REPL.
My first naive approach was to simply replace all \n linebreaks with blanks b/c sclang does not care about linebreaks (like e.g. python), right? Well, having comments in one of the lines will also comment out any following lines with this aproach which is not what we want.

f = {
  // hello
  "foo".postln;
}

results in

f = {//hello "foo".postln;}

Instead of doing some regex hacking I want to go for a clean and proper aproach but this is a deep rabbithole…

Playing around in sclang cli one will find out that there aren’t multiline commands allowed like in other REPLS such as IPython.
But where there is a XKCD for everything, there is also a stack overflow question for everything, and thankfully there is also an answer how to do this, introducing the omnious -i flag to sclang.

A quick look at sclang -h reveals

-i <ide-name>                  Specify IDE name (for enabling IDE-specific class code, default "none")

OK, I do not understand why an IDE name should be relevant and default none sounds like its not really relevant but lets try it out.

With sclang -i foo I can now

*** Welcome to SuperCollider 3.10.4. *** For help type cmd-d.
f={
"foo".postln;
};
f.();^[
foo

where ^[ is the escape key or escape sequence.

Great, mutline sclang REPL - but lets compare this to our non -i counterpart:

*** Welcome to SuperCollider 3.10.4. *** For help type cmd-d.
sc3> "foo".postln;
foo
-> foo
sc3>
  • First, we get a sc3> prepended string representing the prompt which is missing in the -i - it turns out that this is a good feature to determine wheter there is something needed to print or if the prompt is available
  • We do not just get the printed foo but also the return value of the called function as -> foo just like an Qt SC IDE.

But wait - the SO answer states

(this is used for sclang’s unit tests

which leads to https://github.com/supercollider/supercollider/blob/develop/testsuite/sclang/launch_test.py

Maybe this can explain how we can send and receive commands in sclangs REPL in a programmatic way.

Sadly, the unit test is broken, but there is a complete sc test suite w/ equivalent code which also shows how to retrieve code from stdout but does not lift the mysterium how we can properly parse an answer (this only polls for any answer but how do we know the command has been run or if it just does not return anything - something like sc3> is really handy here or are there some hidden non-printable ASCII chars that indicate something I do not see?).

Well, maybe take a look at the C++ implementation of the REPL - and - we can find a sc3> string in the source code - but my C++ knowledge and IDE is not far enough to understand what the -i flag exactly causes and how I could maybe have a mariage of both worlds (-i multiline but non -i sc3 > string and ->).
Where is the code that outputs the string representation of the return value for example (->)? Why is the default file none which sounds like nothing is loaded but why is there a sc3> which is missing otherwise.

There is also a JS implementation which handles communcitation with the sclang process, but I did not fully traced this yet.

If someone could help me or give me hints would be really great, I am sitting since 3 days on this issue and have not made much progress on understanding how the sclang REPL works in -i mode.

My current implementation using the REPL Wrapper of metakernel

class ScREPLWrapper(REPLWrapper):
    def __init__(self, *args, **kwargs):
        cmd = pexpect.spawn('/Applications/SuperCollider.app/Contents/MacOS/sclang -i jupyter')
        cmd.expect('Welcome to SuperCollider', timeout=15)
        super().__init__(
            cmd,
            prompt_regex='',
            prompt_change_cmd='',
            new_prompt_regex="\r\n",
            prompt_emit_cmd="\r\n",
            echo=True,
            *args, **kwargs
        )

    def run_command(self, command, *args, **kwargs):
        return super().run_command(
            f'{command}{chr(0x1b)}\n',  # add escape character
            *args, **kwargs
        )

which execute commands that prints something but is stuck if a command prints nothing.

2

If you are launching SC through an editor, this will also include support classes for interaction between the editor and the language (e.g. ScIDE class). The specific support classes to include will depend on the editor that is launching sclang. So yes, the editor is relevant.

sclang doesn’t print a prompt for editor usage because the user is not interacting with the sclang process through a terminal, but rather through an editor. The prompt would appear in the post window – which would be silly (there’s no good reason to present the prompt in a panel where the user can’t type).

I’m not intimately familiar with sclang’s (std)ins-and-outs, but I suppose it should be possible to define a new -i value and use that to alter SC’s text input/output behavior. That would be a C++ patch on the existing code but I’m quite sure that would be welcomed.

Edit: The -> comes from the class library, not C, which is maybe why you didn’t find it. supercollider/SCClassLibrary/Common/Core/Kernel.sc at fb89a8b6eda01a12593b0976dadc73936628dd0c · supercollider/supercollider · GitHub – this could definitely be made conditional based on the ide name.

hjh

3

The prompt behavior could be hacked like this – probably not ideal but it should give you " -i multiline but non -i sc3 > string and ->" (in Kernel.sc):

	interpretPrintCmdLine {
		var res, func, code = cmdLine, doc, ideClass = \ScIDE.asClass;
		preProcessor !? { cmdLine = preProcessor.value(cmdLine, this) };
		func = this.compile(cmdLine);
		if (ideClass.notNil) {
			thisProcess.nowExecutingPath = ideClass.currentPath
		} {
			if(\Document.asClass.notNil and: {(doc = Document.current).tryPerform(\dataptr).notNil}) {
				thisProcess.nowExecutingPath = doc.tryPerform(\path);
			}
		};
		res = func.value;
		thisProcess.nowExecutingPath = nil;
		codeDump.value(code, res, func, this);
		// hack to output directly and give a prompt
		if(Platform.ideName == \jupyter) {
			res.postln;
			"sc3> ".post;
		} {
			("-> " ++ res).postln;
		}
	}

You could also delete res.postln; if you very strongly object to sclang posting the last code block’s return value – but sclang does this for a reason, so, before you do that, consider carefully whether it would make things better or worse.

hjh

4

Thanks for the replies, they helped me a lot.
I figured out that the best way to know when a command is finished by embedding the command between to print statements on which i can do a regex matching.

{
var result;
"///START COMMAND///".postln;
result = {
   < CODE THAT NEEDS TO BE EXECUTED >
}.value();
postf("-> %\n", result); 
"///END COMMAND///".postln;
}.fork(AppClock);

A python implementation to execute arbitrary sclang code can be found here.

Does this solution has any downsides that I do not see right now (except of course that when someone types ///END COMMAND/// as e.g. print statement)?