I propose that we organize a list of academic institutions that use SuperCollider and that could eventually host a foundation.
As @muellmusik mentioned, I has evaluating Processing Foundation structure which consists of:
1 - Executive Director
1 - Education Community Director
1 - Programs Manager: Fellowships & Programs
1 - Program and Communications Coordinator
1 - Finance Manager
4 - Project Leads
3 - Mentors
5 - Directors (Board of Directors)
9 - Advisors
And now some points that I am trying to estimate:
SC community seems to be 1/25 of Processing community size (measuring by the most viewed video on YT 1M of Processing vs 40k in SC).
Processing Foundation develop several different actions more than supporting a programming language that would be difficult at first for a SC foundation, e.g. fellowships, festivals, etc.
Although Processing maintains three big branches (java, javascript and python), it is wrapped around comercial GPL. The scope of SC seems to be more ambitions since it relies both on developing a new audio server as well as new programming language.
Processing 3 → Processing 4 (mainly a java 11 update, but not only) does not included the same amount of breaking changes that SC3 → SC4 would include.
SC has more contributors on github than Processing 3 (209 x 117).
The initial creators of processing seems to be actively maintaining currently, which is not case for SC.
Many questions here: how different is SC dev workflow from the Processing one? (I have no idea since I am a user).
Would it be possible to bring people that were part of initial developments of SC back? I don’t want to personalize this, but on Github contributor list there are many people that made tons of contributions in the past, but none nowadays. Is it possible to have these people back?
Can we have help from other pro audio FLOSS devs from other communities? Again, I don’t want to personalize, but is possible to have improvements, ports, etc from people like Jatin Chowdhurry, DragonFly reverb, Reaper, Ardour, Jack, etc?
I think a more reasonable governance model might be to develop an institution independent of any specific university, but include major programs that want to give resources or funding in any board / advisory structure. University budgets and curriculum feel too fragile to base a long-term project on, and it would be useful to pull in new institutions as they appear.
Somewhere between unlikely and definitely not. Some left because their interests simply diverged - maybe some of those folks would return, but a good chance they would not simply out of lack of interest. Some left because of fundamental disagreements with the community or bad-faith interactions that made them no longer welcome - a few might return if there was an interesting project where they felt confident they had an environment where they could actually succeed - but most would be very unlikely to come back, and it may not even go well if they did.
On the flip side, I think we have lot of deep expertise in-house that is under-utilized. The “core” dev team is functionally more of a maintenance team, and could probably do more substantial work if some of the daily bureaucratic duties were lessened. I can think of at least five or six people around here who have the experience necessary to take on deep, generational changes to SuperCollider but don’t do so either because they don’t trust current processes for making big changes, and/or simply don’t have the time to do that work in an evenings-and-weekends format. I’d gladly drop my day job for a while and work on big-ticket “SuperCollider 4” style features, but I have to pay rent…
I think organization help would be good. In terms of programming topics, again I think we have most of the knowledge and resources we need, we just need to be able to marshal them. I’ve been keeping a list of all the “SC4” topics in that thread, and we have at least 2 developers around who could confidently tackle any one of those issues if we help facilitate that and, probably, pay them for the hundreds of hours it would take.
I think there is a huge space for incremental change here. An entirely “new” language or audio server is almost for sure unnecessary - but there’s lots of space swapping out one new “piece” of the system while leveraging the stability of the rest, especially if we can resolve some fundamental problems like a general plan for backwards/forwards compatibility, and a better process to make decisions about architecture and API design.
The existential problem is not funding, but governance, and that’s now (at least) the 3rd competent person gone from this project to say that. We didn’t care as much about getting paid as we did about getting things done without suffering procedural abuse.
I suspect a few people might be tempted back if the conditions changed in the right ways for them. Some who left because of bad experiences probably not. Some moved on for life reasons, ie big jobs, families etc changing in ways that didn’t allow the same commitment. Some might be willing to work on clearly defined tasks, and pay won’t hurt of course! But as Scott says, there are very talented devs involved now. So I’m not sure getting people to come back needs to be a main focus.
So I took a look. I don’t know if anyone has more info on this, as there’s little info on their page, but it looks like maybe a lot of individual donations and some support from places like Mozilla, NEA, and partnerships with relevant institutions and education groups.
I wonder to what extent the governance work of the foundation is essentially volunteer? This may be something where institutions could easily be of use, as IME it’s not too hard to justify this kind of involvement for staff. With several involved and a flexible structure that should make things more robust.
The activities go far beyond just dev work. That seems to be mostly facilitated through a series of annual fellowships targeting specific exciting additions (although most of their fellowships seem more focussed on education, outreach, and artistic stuff). How longevity and ongoing technical debt are handled once dev related fellowships are over is unclear.
So I think something like this could work for SC. I think it would be nice to spread ‘hosting’ across lots of sites/partners, and try to keep governance cost as low as possible with in kind labour where that’s feasible and where the people involved have privileged situations. I suspect to attract support this would need to go far beyond dev support in similar ways to PF. That would be a good thing though, and I suspect would improve the dev situation in a number of ways.
I don’t think this is registering with people, and I’d like to expound on my version of it:
One reason I left SC dev is because I saw multiple large-scale positive changes systemically and aggressively blockaded.
For the long-term health of SC, there are numerous components that need to be either entirely removed or at least excised from core and relegate to third-party maintenance.
Because of #1, it is impossible for anyone to push forward a decision to remove a component. (I once lightly proposed retiring the SCIDE and I was immediately lectured about how “hard work went into it and we need to respect that.”)
The scope of SC can only expand, never contract, and maintenance can only become more burdensome.
Development is entirely spent putting out fires that ultimately are products of scope creep. This was a tremendous motivation killer for me as a dev: slogging through technical debt with no power to address long-term health.
Starting a foundation does nothing to address these problems. These are fundamental issues that are largely being ignored in this conversation.
I don’t have any experience with funding open source projects, but the Abjad developers told me that they have never accepted funding because donors always want something specific in return. From what I’ve seen, academic orgs generally want new features, not the removal of old ones, so it could worsen the problem.
I and several other SC contributors have independently concluded that the situation is largely unsalvageable, and the only viable solution is a ground-up rewrite.
A significant contributing factor to why I don’t contribute more to the core. I spend all day tip-toeing around legacy code and technical debt at work, I just can’t justify doing the same in my spare time, for my own sanity. I did this for many years in SC, and I’m happy with a few of the things I achieved, but it wasn’t sustainable or fun.
It doesn’t solve them easily or automatically, but it definitely provides a potential path to solving them. The kind of work you’re talking about - spot fixes to reduce maintenance cost, extracting unnecessary components to external modules, decoupling to reduce dependencies - are totally normal parts of software development. They’re just thankless and boring and hard to build consensus on (you and others are well acquainted with this…), so it’s unlikely anyone is every going to do them as a one-off volunteer project for funsies (especially when compounded with the history SC has of overly contentious discourse around dev topics).
But you’re right that the solution is not JUST to drum up a little funding. The model, especially with something like SuperCollider, is to bring in lower level developers to help out with maintenance and small fixes, in order to slowly build their skillsets and allow people with more domain expertise to focus on larger-ticket generational features. If you can make it interesting enough for new contributors then you have an escalator that can hopefully withstand people leaving once in a while. SC is quite bad for this: the barrier for entry is quite high, and we haven’t done a good job at giving new contributors a feeling that they at least might get to work on something cool or interesting, so we have a bunch of abandoned PR’s and all the people that have the energy and expertise to bigger foundational changes are either burned out or doing things like fixing CI outages.
I think that the creation of a foundation can help with this governance problem. What I am not sure if there is enough people to maintain it.
I understand that there are big problems and breaking change decisions in which consensus is impossible. In this case why not to choose a purposeful divergence model?
I guess Pd faced the same dilemma years ago with PD-Vanilla vs PD-Extended vs Purr Data. Why not to split SC into a “experimental” branch with radical modifications and a more stable one with the traditional development workflow? Maybe this is the most peaceful path?
PS: same thing kind of happened to Jack1 and Jack2… What can we get out of these?
If SC 4 is gaining tracktion one day – not at all sure if this ever happens, as it was suggested already more than 15 years ago – I also assume that this would be the way to go. It would be a shame to disrupt an ecosystem that is right now – by all objections – capable of amazing things that can hardly be achieved with anything else out there. On the other hand, I understand, the need for perspectives is urgent …
Many of the people involved in the repeated hostile behavior (including, in some instances, outright ageist discrimination) towards productive contributors, as referenced by @VIRTUALDOG and myself, remain active in the SC community and have suffered no consequences whatsoever for their actions. We can’t even begin to think about a foundation without confronting this.
I’ve thought this through and, once again, all signs point to a total rewrite being more manageable than a fork.
IMO the worst scenario is to loose brains due to absence of consensus. Disagreement is important and inevitable, however abandonment in the case of such a small community can lead to the fade out…
@nathan, it seems that you, @VIRTUALDOG and @lnihlen would like to propose a new version of SC from scratch, right?
I read on Hadron Postmortem — Lucile Nihlen a complaint about the way users and dev teams deals with novel propositions. So, why not to develop a new version privately and release it when it is mature enough?
I think that missing at least an alpha version of Hadron due to absence of support (financial, emotional, etc) from the community as a whole (devs and users) is such a terrible mistake…
Nathan raised some important points about scope creep. He is totally right that lots of stuff can (and should!) be either thrown out or kept as third-party extensions.
Pd extended (a fork of Pd with “batteries included”) faced a similar problem 10 years ago. It was a huge project and the lead maintainer eventually burned out; nobody dared to take over and the project has been essentially dead for many years now. Purr Data followed its footsteps and I have the feeling that it will eventually meet the same fate…
Pd vanilla, on the other hand, is alive and well. It has a minimal core (some would say, too minimal) with a rich ecosystem and a really nice package manager (Deken). Now, Pd has its own share of problems with technical debt, but it is kind of managable and we are still able to make big leaps forward. For example, the upcoming Pd 0.54 release will have true multi-channel support Of course, it should be mentioned that Pd follows a BDFL governance model (Miller Puckette).
I think the lesson here is: if you want to build an audio programming environment, keep it as small and simple as possible, but provide a rich interface for third-party extensions.
Fortunately, there a exceptions. Pd has been supported by the IEM for decades and I don’t think they have any particular expectations. Finding a generous and reliable(!) institutional partner can be invaluable!
To correct the record, I have never indicated that the lack financial support was a barrier to development of Hadron. I’ve never asked for money, and in fact would not be able to work on Hadron for pay.
Just to add on, as the info might have been easy to overlook by my reference to your paper in the other thread (SuperCollider 4: First Thoughts - #291 by dkmayer): the SC community also got a lot by IEM’s support of your work (VSTPlugin), even if it was a “side-result”.
One example of the risks associated with starting a project from scratch and the potential consequences can be seen in the case of Perl 6. The development of Perl 6 resulted in significant changes and incompatibilities with the existing Perl language. The magnitude of these changes was so substantial that the Perl community decided to rebrand the new language as “Raku” to differentiate it from Perl.
This experience highlights the potential trauma and challenges that can arise when attempting to build a framework from scratch.
Learning from such experiences, it is often advisable to consider strategies that help to mitigate potential disruptions.
An interesting aspect of the development of the new language is that it incorporated suggestions and feedback from the community. The language designers took into account the desires and requests of the majority, who often had specific SMALL changes in mind. However, as these individual suggestions were combined and integrated, the language gradually evolved into something different from what was initially envisioned.
Someone said at the time: "The Perl 5 community, meanwhile, is split between “fck yeah" and "fck you”.
I just want to chime in to say that there are silencious lurkers like me reading this topic. Among them are probably some people asking themselves the same questions you are discussing and observing the situation with the same feeling of urgency that most of you share.
As is the case for most forums, people might feel (this is my case) uncomfortable to join the discussion for they lack a thorough recount of the whole situation and a better understanding of the project’s contribution process and dev practices.
If now is the time to really tackle the situation, I feel that it is important to re-explain clearly why the situation is blocked and how it happened. It looks like this could revive painful memories for some and throw gas on the fire if done improperly because of past conflicts / griefs.
Is there someone on this forum, external to the dev process, that followed the history of the project on the long term? Thinking about somebody neutral that could voice the different perspectives on project lead and dev projects that took place or never took place. It could be interesting for people like me who arrived late in the SC world to understand the different dev philosophies and tensions that arised from the work of all those who tried to maintain / improve this awesome pile of code.
Of course, it should be mentioned that Pd follows a BDFL governance model (Miller Puckette).