from recursive_phrasing:
Pdef can be used as a global storage for event patterns. Here a way is provided by which these definitions can be used as an instrument that consists of several events (a phrase), such as a cloud of short grains. Furthermore, this scheme can be applied recursively, so that structures like a cloud of clouds can be constructed.
without getting into the initial question about different languages, I’d like to express my gratitude to you since the code you wrote revealed to be very instructive and useful to me.
Thanks and cheers from Italy!
kriyananda
I’d just like to add that SCLang would be much more encapsulatable if it had classes in the usual sense. I know that standard class behaviour can be emulated in the language using events and functions but it’s not very natural. This is nevertheless how I abstract things away and get class-like functionality.
If classes, not ones that need to be compiled into the language, but classes that can be defined in the language itself in the usual sense, abstraction as you describe it would become as natural as it is anywhere else I think.
I know I know, but you have to recompile every time etc.
FWIW, I have been using prototype-based programming in SC for about 14 years now.
Prototype object oriented programming uses generalized objects, which can then be cloned and extended. Using fruit as an example, a “fruit” object would represent the properties and functionality of fruit in general. A “banana” object would be cloned from the “fruit” object and general properties specific to bananas would be appended. Each individual “banana” object would be cloned from the generic “banana” object. Compare to the class-based paradigm, where a “fruit” class would be extended by a “banana” class.
ddwPrototype quark: Proto class – The basic concept of a prototype
ddwChucklib quark: PR class – Global storage for prototypes that are to be used like classes (that is, prototypes stored in the PR collection are not to be modified for specific uses, but only copied/cloned)
Protos are a bit slower than standard classes (they use SC environments – environment variable lookup is slower than local variable access – and method calls have an extra layer of dispatch through doesNotUnderstand). In practice, for my work, this has never been a major problem.
(
~operator = Proto {
~a = 2;
~b = 3;
~op = '+';
~prep = { |a, b, op|
if(a.notNil) { ~a = a };
if(b.notNil) { ~b = b };
if(op.notNil) { ~op = op };
currentEnvironment // i.e. 'self' -- must return this!
};
~calc = { ~a.perform(~op, ~b) };
};
// inheritance
~mul = ~operator.clone {
~op = '*';
};
)
~mul.copy.prep(5, 7).calc
-> 35
It’s possible to build large systems this way – almost all of my live coding dialect (ddwChucklib-livecode quark) is implemented in prototypes. This means, if I find a bug in a generator, I can change the code, reexecute the definition, and rerun the pattern-set statement that used the generator, and check the results immediately – no library recompilation.
E.g., here’s an abstract syntax tree, all in prototypes: https://github.com/jamshark70/ddwChucklib-livecode/blob/master/parsenodes.scd
hjh
hi! would you be interested in helping to implement dynamic class library compilation?
I’m almost certainly nowhere near smart enough 
but if I can I will!
right now it’s mostly imagination, but @VIRTUALDOG has looked into the work involved and believes that it’s a multi-person, multi-year engineering effort due to the way classes are handled in sclang’s source code. maybe he can share with you what he wrote down about it — i don’t think it can be practically addressed any time soon, but i think it’s at least valuable to have some eyeballs on how it could be done.
there are a number of practical challenges to working on sclang, which is one of the oldest, biggest, and least generally-understood parts of the SC codebase. it has a really weird architecture largely owing to its original design goal of real-time safety (it ran in the audio thread in SC 1 and 2, as i am told).
if you want to get your feet wet with sclang development, here are two ideas: allowing class extensions and classes to be mixed in .sc files, and allowing var statements to be written in the middle of a function or method. if you could help out with those that would be a great way to get acquainted with sclang and fix major long-term annoyances with the language!
Okay, I think with plenty of time to get to grips with it all I may be able to help, but this kind of thing is a bit of a step up for me so you may receive a number of questions. As long as that’s cool, I’m ‘in,’ as they say.
Something thing to keep in mind also - there’s another version of “dynamically compile-able classes” that also solves most (not all) of the same workflow problems, but is easier and safer: we improve the library compile time so that it’s near instantaneous, via caching and trickery. This is more like a couple of months of work, and is highly testable / far less risky to implement. This means you still lose state when you recompile, but you gain months and months of developer time to work on a different sclang project 
Regarding contributing to either of these “dynamic classlib” proposals: much more of the work required for these is in testing and not engineering work on the C++ internals (I would guess 80% testing/validation, 20% actual sclang internals work) - so there’s lots of room for anyone to contribute to a project like this even if you’re just an sclang user.
Actually, one of the reasons why I wrote Proto back in 2005(?) is because the biggest time sink when recompiling the library is exactly reloading the state that you lost, and not so much the library compilation time.
If I hit “recompile” while working on some music, it goes like this.
So, if I estimate 30-40 seconds to reload the environment and get back to it, library compilation time amounts to 3-5% of that. So speeding up library compilation would really not make much difference for me.
By contrast, by using runtime-defined objects in chucklib-livecode, fixing a bug in one of those pseudo-classes goes like this:
While admitting that Proto is sometimes awkward to use, it does solve a problem (by a factor of 6-8x) that class library caching wouldn’t, for my use cases.
hjh
Here’s a fun C++ experiment for anyone to try, if you want to learn a little about sclang or the possibility of dynamic class modifications: add two primitives (probably to Kernel? but it doesn’t really matter…) to directly call buildBigMethodMatrix and traverseFullDepTree2. Then, call those primitives from SCLang (with as little stack / extra wrapping code as possible). If either one works without completely exploding, then we are very very close to at least runtime modification of existing classes, and possibly even adding new classes.
Apart from some glaring memory management issues, I don’t see any conceptual reason why the method table rebuild (at least…) shouldn’t be runnable at any point, without a full recompile. I haven’t tried this myself - I would very very curious to hear the result if anyone else tries!
Frankly I wonder if an even simpler hack, pre-hooking the perform (in C++) to allow a custom even if fixed-prehook thing before any method is called wouldn’t give us at least a much better Prototype. Basically, instead of only allowing a post hook as doesNotUnderstand there could be a pre-hook that allows hijacking of method calls. Somewhat risky to use of course, but it would allow a much better know. There’s the issue of how to handle exiting such a pre-hook into the normal method lookup, but if I understand the primitive call fallback mechanism, one could simply have a “standard” _perform which when called from this custom prehook would also exit it. So, the default code in Object for this prehook would simply call into that
Object {
performPrehook { |selector ...args|
_perform
}
}
Any custom thing would look e.g. like
Proto {
var ownDictOfMethods;
performPrehook { |selector ...args|
var m = lookupInOwnDict(selector);
if (m.notNil) { ^m.perform(args) } { _perform }
}
}Essentially, this would mean inserting a call to an sclang method into the method dispatch logic itself – and it would have to do this for literally every method call. It’s probably reasonable to expect at least an order of magnitude’s slowdown from that.
hjh
Hmm, what about checking (in the C++ dispatch code) if the preHook method is not overridden from Object’s and skipping the call into it if it’s not? That can’t bee all that expensive and the answer could even be cached per class.
The issue (I have) with Proto right now is that there are hundreds of methods of interest, but Proto only overrides a few, so it’s not a very scalable approach. E.g. I wanted to make a Proto with a custom do, but it turned that (even) do is not overridden in Proto, so not customizable (via Proto’s dict) without subclassing or extending Proto first to override do as a method… and that needed a classlib recompile. It’s true that you only need one recompile per method that you figure out you want to be able to override (in Protos) at runtime. Perhaps some kind of Proto-generator that inspects the classlib and adds every single method might work, but I see issues with conflicting arg lists with that approach…
Honestly, even a better Proto will probably not be very satisfactory because there are over 100 checks of the isKindOfSlot in the C++ code which will fail for Proto-d stuff as opposed to actual sub-classing. Perhaps a way around this would be if for every class in the classlib there was an auto-generated Proto-class with the methods auto-made overridable. E.g. for Symbol you’d get a Symbol_Proto, for Dictionary you’d get at Dictionary_Proto etc. But doing this for all classes could easily double the classlib compile time.
The other possible issue is (and here, I’m not completely certain) that the interpreter may not be reentrant. That is, there are backend functions (MIDI and OSC response functions) that prepare the stack and call runInterpreter, but I tried to do this once from within a primitive and kaboom. I’m not sure it’s even possible for the interpreter, when figuring where to dispatch a message to, to call into the language to do some of the work. Maybe it’s possible; maybe it could be possible with surgery; maybe it’s just not possible.
I’d also suggest that it may not be highly important in the end to have a perfectly transparent pseudo-class. I have more extended reasoning about that but I’ll leave it here for now.
hjh
A couple things -
James is right, adding a “pre-intercept” for all method calls would be inordinately expensive. Simple things like accessing public properties of an object (which currently compile to / are executed as something equivalent to a O(1) method lookup plus a stack push), would all of a sudden need to do a second method lookup, allocate a frame, execute the intercept method, before continuing. For property lookups, this could be orders of magnitude slower even for a one-line intercept function.
isKindOfSlot should only really be used in C++ when reasoning about the slot structure of an Object (in cases where C++ wants to look up a property of an object without perform-ing it via the property getter method). These cases would not / will never work with Proto, because it’s “slot structure” can vary at runtime, so there’s no way for C++ to look up it’s slots anyway. There are better ways to encode these kinds of memory structure details than linking it to a SC type, but even the much more “smart” implementations of this don’t really get you a working, fully dynamic Proto. There are probably many cases where isKindOfSlot is used more freely than is strictly necessary, so it’s not impossible to remove some of these. This would mainly require that no downstream code makes any assumptions about e.g. slot layout, size, etc. of an object - this is not always easy to determine analytically, but may be something to consider if you have specific cases you want to fix.
A much better fit for sclang would be to use the existing doesNotUnderstand hook as-is (since it entails no extra cost on other non-Prototype-y parts of the language). You’d ideally need to:
Object methods whose presence/implementation is hard requirement.Object methods whose implementation is effectively “optional” for Object.doesNotUnderstand. This class becomes your proto, or a base class of your Proto implementation.It is not possible to do [3] without boilerplate right now, but it’s at least possible to do in C++ - and the boilerplate implementation is pretty low-cost. There will be things in list [1] that are annoying and may block the prototyping that you want to do - this is a guess, but I think fixing a handful of individual cases here could be pretty trivial.
You’ll run afoul of isKindOfSlot limitations in the backend - but these, as I said, represent either hard relationships to slot structure / storage or simply overzealous type-checking that can be removed. You’ll also run afoul of isKindOf limitations SC - in cases where these aren’t necessary, removing them can be pretty easy. With the changes in [3] it may be that you can just override isKindOf itself, and embed your own little type system inside of SC. 
On other proposal (I think from Julian, back in the day) was to have a nearly-empty superclass of Object (which currently the library-compiler disallows), e.g.:
NullObject {
doesNotUnderstand { ... }
}
Object : NullObject {
... pretty much everything that is in Object now...
}
Then a prototype class could inherit a basically empty set of methods from NullObject, leaving everything to be “soft-definable.”
Edit: The “empty superclass” would at least have to define *new calling into the primitive that is currently called by Object *new.
hjh
Isn’t that what scztt is doing with his Neutral class (&quark). @scztt how do/did you test that anyway? Did you modify the compiler to use a different base class?