I have two quarks that I’d like to use, but when I recompile my library I receive a duplicate Class error.
Both quarks use the same class name. Looking at the source code, both classes are doing the same thing, though with slightly different code. So ultimately, if I comment out the class from one, both quarks seem to work fine.
Though at that point that quark is listed as dirty in my directory.
How are conflicts like this typically handled, preferably handled? I want to be respectful to everyone involved, and it seems a bit odd to me to arbitrarily pick one and open an issue on Github about it.
And or, I guess does it really matter if the quark is listed as dirty?
Unfortunately, sclang doesn’t have namespaces. As a consequence, Quark authors should really prefix their classes, like it is common practice in other languages without namespaces (e.g. C).
For example, it is really a bad idea to have a class Table in a Quark because that’s an accident waiting to happen.
I think you can raise this issue on both issue trackers by pointing out the conflict with the other library. I’m sure at least one of them will fix it
BTW, you’re lucky that both classes are compatible…
And also don’t be afraid to let quark authors know about conflicts. I for one have thought embarrassingly little about this when publishing quarks and would love to get notified if I screwed up with this kind of stuff
Well I suppose I haven’t done such extensive testing to verify. It’s just that class wise they pretty much do the same thing. However, now that I think about it, another thing that happens is the declaration of a superclass, in both cases a different superclass, and in both cases the superclass is also a class made within the quark… So I suppose things can get weird with method inheritance.
I think I’ll reach out to all the authors to let them know. The class name is aptly named though!
I suppose without some sort of prefix or other type of signifier, it’s really hard to control for. I certainly can’t blame you Mads for not thinking that much about it when making them - I don’t think it’s practical for someone to install every quark in the SC quarks list just to run a check… plus there are others out there not on that list!
This just happened because I was rummaging through the quarks library and being like oo what’s this? and what’s this.
Thank everyone. I just wanted to check in to see if there was already an established best practice for catching this sort of thing.
I did exactly the same thing a long time ago. When I did, I thought that it was because of my inexperience that things got messy and did not work, resulting in all kinds of strange borked behavior.
I see new users doing this now and it has made for some difficult remote deborking sessions.
I have lost faith in quarks and instead provide SuperClean as an Extension. I have not had any trouble since I switched. The install process is much simpler and does not require git. Git turns a lot of new users off, on reason being the prompt that comes up in SC on a mac makes them think of Xcode as an extremely large install. I know this is only Xcode select, which is comparatively smaller, but they don’t know that.
Why use the quark system when there is the alternative of using Extensions instead?
I’m wondering if there’s a way to set up multiple inheritance. In this instance with conflicting classes that basically do the same thing, but are positioned with an entirely different class hierarchy, and so become incompatible, with no easy way to fix that without a ton of work and breaking backward compatibility. This might help resolve these types of issues, though they do seem luckily fairly rare? I installed a bunch of quarks, and I think these were the only conflicting Class warnings I got.
I’m not entirely clear on what the difference between Extensions and Quarks really are. Why do you prefer one over the other?
I like the management features of Quarks.
Git was and still is difficult for me to grasp beyond just a place to go and download stuff from. But recently I’ve been getting a bit more into it, and using a program with a GUI has been sort of helpful for bridging the gap… Though what I think would be awesome is a Git interface that had some training wheels that explain all the terminology in a mouse position aware help browser, like Ableton, or being able to search through the documentation in the same interface, SuperCollider style. The workflow is the toughest part I think, and after learning some of it, I’m finding the command line no more difficult than the software I’ve been using to help manage it, aside from the GUI’s snapshot of the entire repository’s file structure and branches.
The Quarks interface helps hide that in a way that Extensions don’t if you want the ability to autoupdate.
Also, you have very good documentation: SuperClean was one of the first extra things I installed, and it was super easy because of it. But I encountered the documentation at the source of the download link, and the Quarks interface does obscure that. But I found myself clicking on all the Git links anyway to figure out what the libraries were actually all about to see if they were relevant to me or not.
One benefit of the quarks system is that installation and deinstallation happen by adding or removing entries from a json configuration file, and not by copying or symlinking. If the library becomes broken by installing a quark, and the user is working with the IDE, they can go to preferences > interpreter and remove the offending directory, reboot the interpreter and continue.
Also one can have multiple configuration files for different arrangements, without having to manage directories by hand.
You could do this with extensions as well (not required to use quarks for this), but users won’t figure it out by themselves if the extension’s installation instructions say “copy this directory into Extensions.” That’s easier to install but more limited in terms of future maintenance.
In my experience quarks (the actual folders containing the files) do not get deleted when using the IDE + - interface to add and remove quarks. Only the reference to it disappears. This has made things difficult when I’ve helped folks get sat up remotely. If the + - interface in the IDE would work like a remote control for the hidden folders area where the quarks and extensions are at, and it could delete folders there, then I would be more into it.
I don’t want to be all negative here. I like simple things. I like the simplicity of telling folks to drag and drop into the Extensions folder. I am fine with quarks existing simply because other folks prefer it. I don’t have to understand why they prefer it. Different strokes for different folks I guess.
Honestly I don’t know. What I find a bit of a shame is that sclang already has Environments - which would be a natural fit to implement modules! A Quark could store all Classes in a dedicated Environment which itself is stored in the current global environment. This means, classes could be used like this:
var a = mylib.SomeClass.new;
This would be very similar to languages like Python, JS, Lua, etc.
Ideally, sclang wouldn’t just mindlessly compile all Quarks on startup, rather the user can specificy which Quarks they need with something like a “require” or “import” function. This would require that Classes can be created dynamically. Actually, I don’t see an obvious reason why Classes couldn’t be compiled at runtime. But then again, I am not too familiar with the sclang codebase.
I wouldn’t exactly say that sclang mindlessly compiles all Quarks on startup - Quarks are just folders that are added to the include list. The difference between e.g. Python and sclang is that sclang includes are global to the entire project, whereas Python includes are scoped to a single file (of course, this makes all the difference with respect to name collisions…).
Adding namespaces and the ability to shortcut the lookup via imports is a pretty straightforward change. For a quark named MyQuark, all contained classes would implicitly be named MyQuark_ClassName. Then, for code like:
The name resolution for Controller would look like this (psuedocode):
class_name = "Controller:
for (import_name in imports) {
class = find_class(import_name ++ "_" ++ class_name)
if (class != null) { return class; }
}
return find_class(class_name)
Runtime compilation of new classes is not required for namespacing. Runtime compilation is also possible, and the implementation is I think pretty straightforward. The parser-compiler SHOULD be able to compile one-off files and produce a result that could be incorporated into the running sclang. The only tricky thing: adding new Symbol literals, methods, or classes may require that sclang visits every reachable object in it’s runtime, and update the hashes for all symbols and class names. Probably there’s a small amount of work to rebuild the big class-method lookup table as well, but that’s quite global so it easier that updating object in the runtime.
Traversing all objects to update symbol hashes has the potential to be SLOWER than recompiling from scratch, so it’s important to not implement this feature purely for performance reasons - there’s lot of low-hanging fruit around compiler performance that would be much better if the concern was performance.
If an extension has no dependencies and doesn’t have major updates, then there’s not really a major gain to using Quarks (apart from ease of installation). Quarks are only really extensions that specify dependencies, so users don’t have to manually track down ALL of the requirements for a particular quark.
I have different quarks/extensions installed depending on the project I’m working on, and pin these for e.g. a performance, so I don’t accidentally break anything. This workflow is relatively usable with Quarks, but would be an incredible amount of manual labor without the system.
This is a great idea. I’ve been in the habit of prefixing Class names to avoid clashes for years now and can’t believe this never occurred to me. Will give it a try. Thanks!
This is nice! The only problem I have is that it puts the burden on the user and it is not really enforced. (Actually, we have similar problem in Pd. It is possible to manually add namespaces to libraries, but people just don’t do it…)
That’s probably a better way to put it
True. It is not even strictly required for the import mechanism - if it is just about namespaces. However, another purpose of module imports is to only load code on demand.
Usually, the hash of a symbol only depends on the actual string. I can only see this being a problem if objects would store raw pointers to symbols (is this the case?) because symbols might move in memory. However, that also depends on the hash table implementation. “Seperate chaining” generally doesn’t have this problem, but “open addressing” does. IIRC, all hashtables in the SC codebase use the latter, so there might be a problem. But then again, I haven’t really studied the sclang code (yet).
Sorry, let me correct myself - you wouldn’t need to update the hashes of PyrSymbols, obviously these are stable. However, the index field of PyrSymbol, which is used to look up methods in the global method table, would need to be updated any time the global method table changes - I believe this would still require visiting every accessible object.
Ok, I had a quick look at the code and I see what you mean. Whenever you add a new class you basically have to rebuild the global method table (gRowTable).
I believe this would still require visiting every accessible object.
I don’t think this is needed. buildBigMethodMatrix just walks over the Class list and the symbol table. It sets the classIndex field in Class objects and the index field in selectors.
The global method table is a quite interesting optimization to speed up method look up. There is basically no look up chain, just straight indexing into a big array. However, I’m wondering how this compares in practices with the usual approach of using hash tables for method look up. Actually, since SC classes never change at run time, you could just copy the methods of the super class to avoid chained look up. So the difference boils down to a hash table look up VS array indexing (+ the various cache effects).
Anyway, now I understand why sclang has to compile all classes at once. If you would compile classes incrementally, you would have to rebuild the whole global method for each new class, which is near O(N^2) complexity. sclang already uses a thread pool for building the method table, so it is probably not a leightweight operation. I can imagine that compilation time would suffer a lot.
