Strange behavior when using try/catch to return a value from a Class method

@jamshark70 I’ve recently fixed the fact you couldn’t return from try sclang: fix non-local return by JordanHendersonMusic · Pull Request #7449 · supercollider/supercollider · GitHub so that will no longer be the case soon (hopefully).

Giving this thread some more thought I actually think this is a flaw in the language and don’t know how to proceed.

Here is a walk through, because this wasn’t discussed.

A {
   foo { try {^true} }
}

Above the ^ means jump up the call stack until you find A:foo and make true the return value (push it on to the stack).

However, prTry looks like this (I’ve added some debug).

prTry {
		var result, thread = thisThread;
		var next = thread.exceptionHandler,
		wasInProtectedFunc = Exception.inProtectedFunction;
		thread.exceptionHandler = {|error|
			thread.exceptionHandler = next; // pop
			^error
		};
		Exception.inProtectedFunction = true;

		'before this.value'.postln;
		result = this.value;
		'after this.value'.postln;
		result.debug(\result);

		Exception.inProtectedFunction = wasInProtectedFunc;
		thread.exceptionHandler = next; // pop
		^result
	}

Now when we get to this.value, the call stack looks like this…

Interpreter.blahblah
A.foo
Function.try
Function.prTry

Then we enter the function provided to try, i.e. this in Function. When we hit the return, we skip up out of Function.prTry, over Function.try, over whatever else is in A.foo and straight back into the interpreter. The fact the code after this isn’t evaluated is still a bit of a mystery to me, but the error mechanism is borked at this point as the following is never evaluated.

Exception.inProtectedFunction = wasInProtectedFunc;
thread.exceptionHandler = next; // pop

Because we can pass functions with ^ inside them, and they return to whatever method they were defined in, it isn’t possible to check this with syntax, nor at compile time.

The only ‘solution’ I can think of is to emit an error at runtime informing the user this has happened.

The implementation would look something like this…

  1. Add some kind of macro/annotation that adds a flag to the PyrMethod, perhaps like such
Function {
   prTry ##['CannotBeReturnedOver'] { ... }
}
  1. Add this as a boolean flag to PyrMethod.
  2. In returnFromMethod, when walking up the stack, check that no methods that cannot be skipped are in fact skipped and emit a warning if so.

This raises the more general point that in supercollider today, you can do this non-local return absolutely anywhere.

A {
	*foo {
		var f = { ^'returnFromFoo' };
		A.doStuff(f);
	}

	*doStuff { |f|
		'before f'.postln;
		f.();
		'after f'.postln;
	}
}

'after f' is never posted and A.doStuff does not return this as expected by the syntax of the code.

In general, it is absolutely impossible to always do something after calling a function, and once you call a function, you no longer have control over what will be returned from the method.

This means that all methods that do cleanup, can subtly be subverted.


What we need is something like protect, but at the language level implemented inside the interpreter. I am unsure how to do this…

SuperCollider should probably just behave like Smalltalk?

SuperCollider:

sc3> f = { |x| ^x }
-> a Function
sc3> { f.value(3) }.try { "E".postln }
sc3> 3.aMistake
ERROR: 'Function-prTry' Out of context return of value: a DoesNotUnderstandError
sc3> 3.aMistake
ERROR: Message 'aMistake' not understood.

Equivalent Smalltalk:

st> f := [ :x | ^ x ]
a BlockClosure
st> [ f value: 3 ] ensure: [ 'E' printNl ]
Object: 3 error: return from a dead method context
'E'
st>

Best,
R

Ps. Obviously the next mistake in Smalltalk is just the usual mistake:

st> 3 aMistake
Object: 3 error: did not understand #aMistake

Sc had the additional issue that the ^ in command line code returns to interpreter: function compile context, it skips over the printing which is done in another interpreter method.
After which the whole exception mechanisms is broken which is why you don’t get the proper doesnotunderstanderrror.

The question I’m not sure about is how to implement what is ultimately a destructor/defer block that gets called when the frame leaves. The problem is that we don’t have a reentrant interpreter so we can continue up the call stack after executing the first defer block…

Another option, which I’m leaning towards, is to add a return keyword, which only returns to the lexical scope, but allows for inline functions. This way the following would work…

f = { 
   if(true) { return 1 }; \\ exits f
   ... Other stuff ...
}

But…

f = {
   try { return 1 }
}

… Would fail to compile because try’s functions are not inlined. This would allow for a nice compile time error — this is actually similar to what kotlin does.

You could of course also use these in command line code.

This actually addresses this posts original issue, as what they are doing is semantically incorrect, but the return keyword would match their intention and give them the correct error. I think it’s quite common for people to think of ^ as simply being return and for our control structures to be implemented as they are in other languages.

A bit of history – try and protect (IIRC) were a response to a specific issue raised (I’m pretty sure) by Julian, which went something like: “If I use an environment, and an error occurs inside the use function, then it remains in the environment that was used, instead of returning to the environment that was in force before.” JMc said “we need ‘protect/unwind’” and then added this approach.

In fairness, the current error handling mechanism does address the issue that was raised, and it avoided the need to retool any interpreter internals. (JMc was already working for Apple at that time, so if he had made any extensive changes to the interpreter, that would have raised the risk of Apple claiming ownership – he could slip in the couple dozen lines of prTry and still be below the threshold; probably a good thing for the community that he didn’t try anything bigger.)

So we have two features (non-local return, and try/protect) that work within certain boundaries (and served for a long time before people started finding this level of edge case) but which don’t work together.

I don’t have a solution either at this time (and i’m neck deep in a project that’s due for a performance on Sunday) – unlikely I can put much time into this soon.

hjh

Thanks, history is always useful to understand the code.

Try/protect are implemented in terms of non-local returns, they actually work together correctly… It is just very confusing semantically and not what most people expect as non-local returns are seldom used in other languages.

Interestingly, you can also implement non-local returns if the interpreter supports exceptions. The two are isomorphic.

This is problematic because while we could implement the defer bit of try as a c++ function which we could detect skipping over and run when needed, it doesn’t address the general point that you really want to be able to specify this in the language. Consider this (I forget the exact signature of this but you get the point)…

A{
   foo {
      File.use(...){ ^nil }
   }
}

Here the file is never cleaned up.

This is why I like the idea of introducing a new return keyword that only returns locally. Something like…

A{
   foo {
      File.use(...){ return(foo) 1 }
   }
}

Here, you’d get a complier error because the function cannot be inlined into the scope of foo. Perhaps break is a more apt keyword? This way it only works with inlined control flow methods, which never require cleanup.

And just to pick up on this…

This line of code is valid syntax, this means the compile has to run it to find out if it ‘works’ or not.

var foo;
foo.someThingThatDoesNotExist

Whereas …

foo.someThingThatDoesNotExist

… is not valid syntacially, and the compile knows it will not work, so it doesn’t even bother to run it. This is because it will not be able to find the variable foo (missing the var foo bit).

In your example

(
a.foo; //no error or syntax issue reported
didfdifhdjif; // ERROR: Variable 'didfdifhdjif' not defined. (as expected)
)

The compiler will not return any valid bytecode for the interpreter to execute, so nothing, including a.foo, will be evaluated.

It is the difference between a compiler error and a runtime error