Hi all,
This thread is for anyone looking to help out with the class library development… all are welcome, say hi!
FYI: I’m not in charge, just started the thread!
4 Likes
Fantastic! Thanks for jumping in - excited to get this party started…
Hi!
Not sure if/what I will be able to contribute, but I am keeping an eye on the thread.
I’ll join in on this as well. Thanks!
Hallo! Looking forward to contribute where I can and learn lots from the rest of you!
Happy to be involved when I can. Hoping to get back to being a little more involved in SC dev activity.
1 Like
Hey, I would also like to contribute 
I am primarily interested in fixing bugs, doing some maintenance and improving the docs.
Generally I would also be interested in making some implicit takes more transparent and try to figure out where beginners struggle and try to improve the standard library in this regard.
We should probably decide how we wish to communicate?
I quite like the forum, as its open for everyone to see.
Also, I would like to put forward a proposal (or rather draft of one) for consideration. It might give us something do to in the first/second meeting and be a good opportunity for new people (like me) to see what the process of committing changes, getting feedback, etc., would be like and how all this might work?
Proposal
I don’t think the contents of this should be commented upon in this thread. Its just here as an example for something to talk about.
I’ve written some code that inserts thread blocking between accessing member functions. This can be used to automatically ensure a server resource (like Buffer) has been completed before it is accessed, eliminating the need to call s.sync, which I think is a large source of error amongst new users. This can make callback function unnecessary, like Buffer::get(index, action), as the value is just automatically waited upon use. Its a kind of Future I guess.
Here is what it looks like in use.
~path = "...";
s.waitForBoot {
var b = Buffer.readAsResource(s, ~path);
b.query // automatically waits for the buffer to load
}
(
~b = Buffer.readAsResource(s, ~path);
~b.query // this line throws...
// ERROR: Buffer's value has not completed, either use it in a Routine/Thread, or, literally wait until the resource has loaded and try again
)
~b = Buffer.readAsResource(s, ~path);
// twiddle your thumbs for a few milliseconds
~b.query // works
Maybe discussion about proposals can be held on github, possibly using their “discussions” feature (which can be enabled in the repository settings)?
1 Like
Not quite sure where this fits in, hopefully it’s not off-topic here, though it’s a bit of a “meta” question.
I’m curious about the “culture” (for want of a better word) around libraries.
For instance, CondVar seems to have been added to the “core” libraries in 2021, and I’m curious why?
It doesn’t seem to require any language primitives? It doesn’t seem to be used anywhere else in the “core”? It seems like it’s just an ordinary library?
It’s used by UnitTest, which seems to have been added in 2017, but then UnitTest doesn’t seem to be used anywhere at all in “core” and it doesn’t seem to require any language primitives, so it’s just an ordinary library too?
The Quarks system seems to be from 2015 and it seems to be the mechanism for writing libraries?
Why aren’t UnitTest and CondVar libraries?
And also why weren’t any of the “sub-libraries” in “core” that pre-date 2015 and that seem like they’d probably work as ordinary libraries (PatternLib, JitLib, ScDoc &etc &etc.) made into such libraries?
There would seem to be a lot of (I think obvious?) advantages to doing this.
What are the disadvantages?
Are there problems with the current library architecture? (I’m very old fashioned and just use the includePaths/excludePaths system.)
The “everything in one library” approach is nice for small systems with few participants, but it’s not clear it scales particularly well?
Is the “tiny core with many libraries” approach worth considering? I.e. where the expectation is that anything that can be implemented as a library is implemented as a library?
Is there a sense that people don’t want to use libraries? If so why not? Would moving some of the things that everyone uses (Server, UGen &etc.) into libraries help with this? (Obviously the standard installers would include such libraries.)
I’m sure there are complications in all directions!
1 Like
If we could overhaul Quarks a bit - perhaps into something that allows for includes or something like a dependency manager - I’d actually prefer a smaller set of base SCClassLibrary classes.
4 Likes
UnitTest (and CondVar) are used in the automated builds for testing so to me it makes sense to put it in the core lib.
I don’t think there’s a special resistance to using quarks, but there’s sometimes a bit of a problem of discoverability - not everyone submits their quarks into the official quarks repository (I’m guilty of that) and then it can be harder than needed to find out about their existence.
In part this is because until recently I had no idea how to submit it (documentation perhaps could be improved a bit here, or perhaps it just shows how lazy I am for not reading some existing document), and in part it’s because I feel my quarks often remain in a state of evolution which means I’m not prepared to maintaining a long-term stable API (even if in practice I only seldom make breaking changes, although it has happened not so long ago for the Panola quark), which means I feel it doesn’t belong in a list of “official” quarks where I value backward compatibility.
When you install a quark it doesn’t remain a “library” - it becomes part of the supercollider eco-system on your hard drive. You never explicitly import “Panola”, it’s just there. At present, I don’t see any practical distinction between having something as part of the core or having something as a quark that is distributed with the official installer.
One thing I find missing for quarks (actually for sclang in general) is the lack of a namespace feature, to avoid naming clashes which can lead to some weird errors.
Condition has always been in core, along with Semaphore.
Probably around 2021, I proposed a fix for Condition, for true timeout support. The decision at that time was that Condition wasn’t a real condition variable, and that this would lead to a variety of structural problems down the road, so it would be better to add a real condition variable class, as a replacement for Condition. Since Condition is in core, it made sense to put CondVar in core.
Of libraries, I wouldn’t have a problem with moving some feature sets out of core (even patterns). However, while I can agree that SC’s core library is too big, I’m also not fully convinced by Pd’s concept of minimal core. If you have to install externals to get a decent sounding set of basic filters (sorry, vcf~ doesn’t cut it), that’s a bit dogmatic to me. (Took me ages to find the ELSE library.) Somewhere in the middle is the better answer.
CondVar is an interesting case though – every user, sooner or later, will run into thread sync problems. CondVar and the derivative classes Jordan has been working on are a solution to problems that, when you start with SC, you don’t know you’re going to have. To me, it makes sense to have these available without user action.
hjh
Definitely in favour of a smaller class library. JITLib on particular could be moved out. There is also a bunch of old unused code floating around.
What annoys me about quarks is that they don’t have descriptions, and when you go to the GitHub page, most people’s quarks don’t have a readme either. Some stricter requirements might be needed before quarks are added. I’d be in favour of kicking all those that don’t meet the requirements out (after a request to update the description/documentation). There could be some really good code in there, but personally I’m never going to install something with an indecipherable name and no complete description, doubt I’m alone.
Regarding synchronization stuff, supercollider deals with asynchronous relationships in almost every action the user makes because of the server. For that reason, I think this should be baked into the core library in a far more substantial way, and all classes should be more aware that something might wait. One example, if you wait in the .asString method you break the interpreters ability to post window, which isn’t It’s an unreasonable thing for the user to want to do.
1 Like
Can’t see how you would implement this without altering the c++ language code. Would love this, but it’s quite challenging.
One alternative is if the Quarks installed into a project local directory and scide included that folder when it loaded. This does move away from the file based approach, to more of a directory based approach. Personally, I do this anyway, but many people don’t.
Another useful addition to quarks would be the ability to search for class names and/or keywords.
IIRC, Pure Data’s package manager “deken” lets you search for object names, such as, search for “gate” and one of the search results should be cyclone.
hjh
One glaring issue we haven’t discussed is our position on changes that breaks:
- people’s supercollider only code,
- supercollider code that uses things we consider backend
- quarks
The issue with the second point is that there is no private implementation and all the ‘backend’ is actually public, meaning any changes will break something.
We have an ‘ObeaseObjectProgramming’ model where the smallest thing possible in supercollider (Object) is overburden with too much responsibility. Each method in there could be used in all objects in all code.
If we do go down the route of separating things into libraries and decoupling/refactoring stuff - which will break something - how do we deal with this? Do we have a responsibility to be an archive of work, as well as a tool? To be a piano and a historiy instrument museum?
I’m just concerned that we won’t be able to do anything (even splitting things into libraries) if we fully commit to the latter, whilst also don’t want to alienate the userbase. Perhaps some of what has already been discussed might be better under an SC4 name? Or perhaps we have a parallel class library that the user can opt in and out, with the notice SC3 will not be updated? I do not know… if anyone has any professional experience managing such changes please do jump in
The ‘tradition’ in SC is to allow substantially breaking changes for major new versions, which have usually been fairly big rethinks. An SC4 would be great, and would definitely be the time for major cleanup, but would need a large amount of commitment and vision. It’s been discussed for years, with lots of contradictory proposals, so I think it can only happen if one or a few core people really commit to a coherent plan.
Regarding breakage: I think moving some things into Quarks is okay, as long as there’s a reasonably painless way to get that stuff back. eg a download package that included JITLib. We don’t want people to have to hunt around a lot to get back to their status quo.
Regarding async stuff (slightly tangentially): Some years ago I proposed an approach which would essentially move ‘allocators’ into the server. When creating a new Buffer object the lang would assign a UUID which the server would use to identify the object it assigned. This would vastly simplify multiclient situations (which despite being a supposed strength remain awkward), and remove a little async pain in some cases.
Although I am always in favor of deprecation through an improved implementation, I think SC would loose many users by making their custom written code, compositions and beloved extensions unusable.
Already something as “easy” as installing a Quark often results in frustration because it relies on a git which is not a very easy installation on Windows.
A heavy transition will also deprecate lots of learning resources.
As long as there are unit tests with a good coverage I don’t see the problem in having a big code base, and I think NodeJS ecosystem is a perfect example what happens if the standard library is not powerful enough vs having batteries included such as in Python3. So maybe there should be a focus on writing more tests for already existing code?
Some deprecation/update examples are pd extended, see https://forum.pdpatchrepo.info/topic/12797/pd-vanilla-vs-pd-extended-why-such-a-mess as well as the python2->3 transition
And there is also the infamous Torvalds quote “we do not break user space” which seems have been proven beneficial by time.
It will be also interesting how Max MSP “vs” RNBO will play out over time? But Cycling74 has probably enough resources to support both applications for a long time.
The question should be which benefits could justify breaking the current code base? I for myself don’t see any kind of feature which would justify this.
edit: Also after working some years in bigger projects I learned to love the mono-repo which just makes life easier: No version-clashes, easy CI/CD, easy testing, easy update procedures
2 Likes
Supercollider does not have non-user space because there are no private methods. Every method is available everywhere at all times. With that in mind, I don’t see how it is possible to make any changes, only add new features, without breaking something.
Breaking everything and implementing access specifies might allow us to never break anything again… that might be worth it.
I think what @muellmusik suggested was that a ‘painless’ breaking of user code might be a pragmatic metric.
As a non-windows user, I always assumed supercollider came with git - if the install process is painful then this is definitely unacceptable based on the idea of painlessness.
2 Likes