So, just out of genuine curiosity, just throwing this question out there. Is there any chance that some of the SCPlugins might end up in the ‘core’ SC? (This might have happened without me noticing it). Or are there good reasons why this is not happening.
Most of them are really useable and from my experience really stable.
An example: Why isn’t something as like PV_MagBuffer or PV_FreqBuffer (from JoshUgens) included in the core from the get go?
This is a sensible idea on the surface, and comes up often. This is, however, the opposite of the current direction for several reasons:
Having a monolith package like sc3-plugins requires that we release an entire new version of the entire plugins package just to release e.g. one bugfix for a single plugin. And, conversely, it’s impossible for a user to take a bugfix for one plugin without taking on ALL of the changes in sc3-plugins.
For every relatively maintained, organized, and working plugin in that package, there are 3 that are unmaintained, broken, or in a totally unknown state. Separating the “good” ones from the rest is a huge amount of work, and quite simply no one has done it (if someone DID make a “gold star” list of sc3-plugins, this would be useful).
There are currently not enough developers / developer time to properly maintain the core SuperCollider package, so introducing MORE things to maintain will be problematic. Plugins should have owners / maintainers, and if they HAVE no owners or maintainers, this should at least be clear to people installing/using them (in a “beware, proceed at your own risk” kind of way).
Ideally, all of the plugins in sc3-plugins would be broken into separate packages that could be easily installed by themselves, via the Quarks system (there are some small issues with Quarks currently preventing this from happening, but it’s otherwise a reasonable proposal). If that were done, there’s no reason why someone / anyone couldn’t create a “best-of-plugins” quark, or a “plugin starter pack” quark, so people could still install a grab bag of useful things if they wanted.
This is a sensible idea on the surface, and comes up often. This is, however, the opposite of the current direction for several reasons:
Having a monolith package like sc3-plugins requires that we release an entire new version of the entire plugins package just to release e.g. one bugfix for a single plugin. And, conversely, it’s impossible for a user to take a bugfix for one plugin without taking on ALL of the changes in sc3-plugins.
For every relatively maintained, organized, and working plugin in that package, there are 3 that are unmaintained, broken, or in a totally unknown state. Separating the “good” ones from the rest is a huge amount of work, and quite simply no one has done it (if someone DID make a “gold star” list of sc3-plugins, this would be useful).
There are currently not enough developers / developer time to properly maintain the core SuperCollider package, so introducing MORE things to maintain will be problematic. Plugins should have owners / maintainers, and if they HAVE no owners or maintainers, this should at least be clear to people installing/using them (in a “beware, proceed at your own risk” kind of way).
Ideally, all of the plugins in sc3-plugins would be broken into separate packages that could be easily installed by themselves, via the Quarks system (there are some small issues with Quarks currently preventing this from happening, but it’s otherwise a reasonable proposal). If that were done, there’s no reason why someone / anyone couldn’t create a “best-of-plugins” quark, or a “plugin starter pack” quark, so people could still install a grab bag of useful things if they wanted.
Counter-point: having one (or two) sets of plugins keeps a ton of chaos at bay.
If somone publishes a SynthDef which refers to “PlayBuf2”, which PlayBuf2 is that? Can Quarks allow conflicting names? Of course, Quarks refer to plugins, so then you have quarks which depend on certain versions of a “plugin Quark”. Now you need more of a package manager than we currently have, and more of a centralized source repository, than we’re already maintaining.
IMO, point #1 assumes instantaneous fixes. If instead we do regular releases, as we do now, fixes won’t come out as quickly but we maintain stability and developer sanity. If plugins become cheap-and-easy to mainstream, are we ready for that? In the extreme, are we ready for security issues from third-party developers, namespace conflicts and turf wars, etc.? Are we ready for leftpad-style problems with third-party “beest-of-plugins” Quarks? Will there be any hope for playing pieces years after they were written?
Point #2 – again, imo – seems worse without a blessed inner and outer core of packages. Currently we can pull good plugins into core after deliberation. I’d really support that.
tl;dr IMO two blessed sets of plugins makes a ton of sense as a common set of nouns for our small community. We shouldn’t prevent the natural process of some plugins moving from the minor leagues to the major leagues.
This is a sensible idea on the surface, and comes up often. This is, however, the opposite of the current direction […]
Reflecting further, it seems like there are two separate ideas here:
“There should be no blessed ‘outer core’”: i.e. we should split sc3-plugins out into Quarks, etc. I disagree with this view, as noted above.
“There should be no blessed ‘inner core’”: i.e. we should move UGens out of core SC as much as possible, and prevent new ones from coming in. I really disagree with this view. IMO SC makes no sense without a common language of core UGens we can all use and understand. If an experimental plugin is mature enough to join the core, we shouldn’t block effort to put it in.
We should absolutely have a blessed set of plugins. sc3-plugins is ABSOLUTELY not blessed, it’s just where everything got collected when build processes and tooling was hard and needed to be centralized, 10-15 years ago.
I think we’re on the same page in principle, so I don’t wanna argue too much - maybe a few things for perspective:
Code should probably live close to the people / organizations that have knowledge of and maintain it. Pulling something into the main repo implies that those owning that repo have a level of authority over it, and need to have the knowledge that is requisite to that authority - they control bugs, code review, releases, etc. If core devs aren’t able, or capable of, maintaining or making fixes to that code - it shouldn’t go there (I don’t mean “capable” in a derogatory way - there’s lots of research code in plugins that would not be immediately obvious to even v experienced programmers - it’s almost all highly specialized). Given that maintaining the core is already at capacity - and water spills out of the bathtub the moment you put anything else in - it seems like there should be a good argument for why any specific plugin should be part of the same commit history and version semantics as the e.g. the class library. There may be plugins that fit the bill - I’m not sure what they are. None that i know of - even the useful ones like the BEQ suite! - really qualify in my mind?
In a practical sense, I doubt that anyone actively making and maintaining plugins NOW wants to wait for pull request approval from anyone working on class library maintenance, or wait 1-3 years for a major SuperCollider update to push a minor bug fix.
There are lots of ways to have a blessed list of plugins that don’t require a shared commit history, versioning, and community code review and development effort. If the UX need is simply that a non-pro user can click a button / run a line of code and get a reliable, tested, known set of plugins on their system - this is a very valid and good problem to look at. sc3-plugins is NOT that at the moment, and I think “how could it become that” is a very good question. Re other threads elsewhere regarding participation: a massively valuable task that anyone could do right now, that takes very little specific expertise, is to go through ALL of sc3-plugins and test + document whats there, what works, what doesn’t, what’s weird and unknown, when things were last updated, who wrote them, do they have helpfiles, etc.
There are latent namespace and security problems, I agree. Honestly, I would LOVE it if we had namespacing problems with PlayBuf, because there were so many fantastic and useful PlayBuf versions floating around that we had to choose…
I think the vectors along which to split this decision:
commit history (is it important that a commit to file X be in the same history as commit Y)
code review / management (are the same people / decision-making processes overseeing changes to code X and code Y)
versioning (does a version change to X necessitate a version change to Y)
Technical debt pragmatics (e.g. because of tooling stuff, it’s easier to have X in the same project with Y)
I don’t myself see a lot of reason for UGens to share the same repo as, e.g. the ScIDE for any of the above reasons other than - they are already there and it’s a pain to change - but maybe it’s worth brainstorming what the benefits might be.
Based on my recent involvement in PRs, I should say that including the SC3 plugin in the SC3 installation package seems very difficult in terms of the review, approval and merge process.
However, from a simple user experience point of view, installing individual packages separately is very clumsy. From a pure user perspective, the best way to get all the useful plugins and quarks would be to simply install an official SC install package that includes them all.
This is actually a very sad thing, at least for users like me.
+1. I don’t see what this would be clumsy. It’s all there for you to cherry pick. Other programming languages list the date of the last release and how many imported the package. This could give a idea how useful the plugin is. Projects could provide ‘metapackages’ (metaquarks) which installs some plugins maybe.
Unlike other programming languages, there is no import in sclang.
Thus, evaluating some code without commenting on the class included in the core SCCliassLibrary sometimes leads users to research the classes used in that code. These questions can be found on the current scysnth forum and the old mailing list.
Would it be better to make import mandatory at the beginning of a block of code when using classes outside the SCClassLibrary code?
p.s.: I always insert "".include when using a Quark.
There is this feature, and it includes the commit hash, so it is deterministic.
I never tried, and I was told it never worked.
So, it comes down to someone checking in good code again.
How PD deal with the libraries in which the authors are not around anymore? Do you require a maintainer for every package? Do people show up in enough numbers?
Just like in any other language. Either the library is stable or you hope that someone will come and maintain it.
do require a maintainer for every package?
Someone needs to be in charge of the repo, but anybody can upload binaries for new platforms. Many plugins are basically archived, but sometimes people fork the repos and release new versions.
do people show up in enough
Generally, yes.
Note that you have the exact same problems with a huge monorepo like sc3plugins, you still need people to maintain the individual plugins.
Security risks were ever brought up as a discussion?
I ask because security is a hot topic again after the HIGHLY compromised xz package was accepted in Debian unstable. A guy working for Microsoft discovered the backdoor by chance (real-world irony).
EDIT: (this is more relevant to PD than SC, so maybe just ignore this question)
I think the question of which repo core UGens live in is pretty secondary; my primary question is which UGens are available when you click “Install SuperCollider,” run apt install supercollider, or whatever.
Can I write a piece that plays on your computer? Right now we’re more or less all “speaking the same language,” and I’m concerned that package managers quickly turn the situation into “well, it works on my machine.” We’re also unfortunately mostly speaking a language that was mostly defined in the late 1990s: I don’t recall (m)any new additions to the default-install UGens in a long time.
My criteria for UGens that could ascend from whichever outer core (either sc3-plugins or a 3rd party repo) into default installs is:
High quality
a. Well-written
b. Sounds good (subjective, of course)
Trustworthy
a. Won’t crash scsynth/supernova
b. Won’t steal my credit card info
Unlikely to change in the future
Point #3 is both for the “speaking the same language” point above, and to minimize the maintenance burden for the core maintainers. A motivated person/group would need to make a thorough case for a UGen’s inclusion based on the criteria above, so that the main job for the core maintainers is an accept/reject based on the proposal.
Honestly, I would LOVE it if we had namespacing problems with PlayBuf, because there were so many fantastic and useful PlayBuf versions floating around that we had to choose…
Same! I think my view is that in the absence of one fantastic version we’ll have several conflicting works-most-of-the-time ones. Also: if there were one fantastic PlayBuf improvement, wouldn’t it be good to have it in SC core?
I do see real advantages of a Quarks-based install (e.g. more PlayBuf experimentation could happen easily, and iirc my Ambisonics install would have been easier!), but I think we should consider the downsides and aditional required maintenance, and not block inclusion of UGens which really do belong in the core.
There have been concerns, but in practice, I don’t recall a single instance of a compromised Pd external. Pd and SC are a tiny niche and don’t really make sense as a target for hackers or other malicious actors. FWIW, Deken packages may contain GPG signatures.
… of course, this is up for discussion too. (Probably not here, though!)
In my ideal world “the core” UGens would be very minimal, and installing / swapping out UGen sets for different use cases and projects would be very trivial.
There have been concerns, but in practice, I don’t recall a single instance of a compromised Pd external.
…that you’ve found out about. Binaries are really dificult (and sometimes mathematically impossible) to verify and very few do. Malicious code rarely announces its presence.
Pd and SC are a tiny niche and don’t really make sense as a target for hackers or other malicious actors.
This viewpoint seems to massively overestimate how hard it is to produce a malicious binary. Someone could modify a UGen to do something malicious in a couple hours. Just two of many possible motives: a person with an unusual sense of humor playing a prank, or a developer who’s angry at someone else in the SC community. Not to mention people who compile using a compromised toolchain, etc.
