Yeah so immediately I think this all sounds great, but I wanted to ask if there are any RT safety/performance issues? I can think of lots of use cases (large scale number crunching, pre-optimising demanding functions that will be extensively reused), but of course part of the reason that SC performs the way it does is it’s designed for RT. I know people have struggled sometimes with that aspect when implementing other SC clients, especially within general purpose languages. Anyway, this sounds very exciting, Luke!
So far, for my own limited use, I have no requests for new features in supercollider.
I am often upset by software projects that decide to make some sort of great leap forward, instead of incremental improvements. I highly value back-compatibility, i.e. old code should always continue to work. One of my heroes in this is the TeX typesetting language, current release is 3.141592653. A possibly more relevant example would be javascript, where a lot of features have been added over the years without breaking anything old (as far as I know, I’m only a casual javascript user). There is some discussion of improving the documentation in the previous comments, which I think is very important.
In summary: my vote would be for an effort for complete documentation, coupled to comprehensive testing of examples from that documentation that would serve as a permanent compatibility reference.
2 Likes
I’m interested to see this happening too. I hope it goes very well!
And yes, careful attention to latency seems to explain many aspects of SuperCollider’s design.
In some contexts latency seems to require multiple compiler passes?
(C.f. the “new non-optimising JavaScript compiler”, https://v8.dev/blog/sparkplug)
I’m also a little bit curious how the sclang argument/arity semantics, which seem intricate to me, affects making things very fast?
f = {arg x=1, y; [x,y]}
f.value == [1,nil]
f.value() == [1,nil]
f.value(3) == [3,nil]
f.value(x:3) == [3,nil]
f.value(3,2,1) == [3,2]
f.value(z:1,x:3) == [3,nil]
-3.abs == 3
-3.abs() == 3
-3.abs(1) == 3
-3.abs(x:0) == 0
-3.performList(selector: 'abs', arglist: [1,2,3]) == 3
I know the usage is very idiomatic:
[1,2,3].collect({'x'.postln}) == ['x','x','x']
[1,2,3].collect({arg i; i * i}) == [1,4,9]
[1,2,3].collect({arg i, j; i * j}) == [2,4,6]
[1,2,3].collect({arg i, j, k = 3; i * j * k}) == [0,6,18]
Is this a non-issue for a compiler? (Lisps are a little bit like this and they can be very fast!)
Best,
Rohan
Ps. I notice .perform is like .if about keyword arguments, it doesn’t like them…
-3.perform(selector: 'abs') // => error
This is a bug for sure – seems like the failure to match the keyword arg doesn’t pop values off the stack, when it should.
hjh
I have… an idea, which is more of an ideal vision, of how the docs could be organized:
Client Server SC X
- Client: Core, Scheduling, Live Coding, Utilities
- Server: Bus, Buffer, Ugens, Server
- SC: Development, Platform, SC4, Walkthroughs
- X: Quarks, Plugins, Interfaces, Qt/GUI
…refined to ensure all existing doc entries have there place
Thoughts?
Your work on Tidal documentation sounds fantastic! I also echo your and several other people’s sentiments here that a documentation overhaul does not need a version 4.
There is a tutorial available on how to contribute to the documentation on the SuperCollider github wiki page: The entire Developer Reference > General Workflow is great, with specifically the Creating Pull Requests section describing branch naming conventions, as well as formatting pull request messages, etc. Creating pull requests · supercollider/supercollider Wiki · GitHub.
Eli Fieldsteel also put together a great tutorial on this:
It might be good to break out documentation refactoring into a new thread. There’s the one that @Rainer started recently, though seems geared toward a few specific pages-- Writing Help: Workflow, or we can start a new one specifically for refactoring and reorganizing on a larger scale…
1 Like
So I discovered something interesting just now, actually navigating the help documentation for your plugins extension. In the help you link to your github page, and well, the link works, and the IDE help browser displays github, and markdown. Navigating to github links found at the bottom of the page was also possible. Likely not helpful on its own, I just happened to notice this today, and thought of your post. Not sure if others know about this.
2 Likes
I would love to see SC4 be documented in markdown.
I’m not entirely sure SC4 is a good idea, but I do think making SC3 nicer is an excellent one!
So, if people are working on the help system, and are collecting thoughts, here are mine:
Would it make sense to implement more of the usual the Smalltalk infrastructure?
It’s not very complicated, and it provides a super clear design model to follow.
Also I think some aspects of the current help system are a bit confusing.
For instance, how it treats all the classes the same way.
There are lots and lots (and lots) of UGen classes, but they basically all have the same structure and follow the same protocol.
It doesn’t seem to make sense to document SinOsc and String (for instance) using the same template.
I think it might be clearer if the “Language” documentation lived in the class/method tree (as in a standard Smalltalk system) and was accessed using standard “Browser” and “Finder” tools.
Then the “Help” could be a separate system altogether, “Help” files being more like little essays on what something does and how to use it.
At which point, of course, you could just resurrect the old SC3 help system!
For people who don’t remember, SC could formerly read and display and edit rich text files.
So as well as ordinary plain text .scd files you could open .rtf files, in the editor.
The help files were just .rtf files that followed some simple conventions, and lived in a known place for looking up.
You navigated the Help system like a sort of implicit wiki, following references to other help files.
Because they were just ordinary files in the ordinary editor all of the ordinary editor commands worked in them, for running and editing examples, and so on.
Also you could annotate the help files as you liked and save the changes.
It was a very nice literate programming system!
I miss it, it would be lovely to have it back…
Markdown is nice too, but it requires an edit/compile/render/view cycle, and a mode switch.
It seems QT has a rich text editor now? So the old style might be feasible?
My understanding is the old system was discarded because it was too hard to get it to work anywhere other than on Apples.
Maybe things have changed?
3 Likes
Very much agreed.
Regarding the help system, I’d personally love to be able to work in some variety of markdown, since I already use it for my internal project notes, but I haven’t really thought through the larger scale implications of that at this point.
What I actually wanted to add here is the idea that one of the simplest and highest-leverage changes that could be made to the help system would be adding full-text search. Right now we have options for “Title/Filename”, “Summary”, “Categories”, and “Methods”, and I’d imagine this just being another checkable item in that list.
I think this would lower the barrier to learners building their own mental maps of the documentation (which is complex and fragmented and realistically will probably always be at least a little complex and fragmented). Speaking personally, before I realized that I could just grep -ir foo /usr/share/SuperCollider/HelpSource, I’d regularly have the experience of stumbling across something interesting buried in some corner of the help system and then “losing” it for months before something else happened to lead me back to it.
Completely agree, IMO the old (SCApp) documentation system was a much better solution. Sure, there’s more consistency in the appearance of the documentation now, but it came at the price of flexibility and interactivity.
Unfortunately .rtf is a Microsoft proprietary format, and there doesn’t appear to be any libraries for handling that particular format cross platform.
It would be great to have an environment more in line with the old SCApp experience in general, which in itself was closer to the SmallTalk ethos.
NOTE: no disrespect to any authors who’ve graciously contributed to the development of SC 
Its searchability was poor.
hjh
Indeed, I’m probably overlooking weak aspects of the old implementation, it’s been a while since I used it.
Unfortunately
.rtfis a Microsoft proprietary format, and there doesn’t appear to be any libraries for handling that particular format cross platform.
Yes, .rtf wouldn’t work. It looks like the QT system would store documents either as HTML or “a custom XML format”.
https://doc.qt.io/qt-5/examples-richtext.html
But I’m not sure it matters what the storage format is?
Another complaint about .rtf was that diffs were hard to read. Maybe using auto-tidied HTML would help there too?
Its searchability was poor.
Also, it did all the kinds of documentation the same way too. ArrayedCollection.help.rtf was the same as COsc.rtf.
But perhaps all the method documentation in the former is more helpful if it’s written in the methods (as “Documentation”), and the latter (which only documents one method) is better as a “Help” file.
In case anyone is interested, you can grab an archive of the old .rtf files by typing the below, if you have a copy of the SC git archive:
$ git checkout 2495f4f61c0955001169d28142543aabbe1bedcb build/old_rtf_help.tar.gz
(I’m sure you can do it at github too, but I don’t know how.)
SCDoc has a lot of structure and a lot of tools for navigating and transforming it (that’s how it works!) so if people did decide to make any changes they could be done using the existing help codes.
One thing I don’t miss from SCApp: language crashes, you forgot to save, bang your code is lost forever. Or, you’re using the internal server for scoping (no shared memory interface at that time), and a UGen crashes, bang lost your code.
I’m curious what other features you liked about the old app. Rich text for code, unstructured editing of help files are two that you’ve mentioned… anything else?
hjh
Since I mostly use my own client, I have no strong opinion on the direction of the sclang help system, but I would recommend against going back to RTF. It’s also not true that markdown requires an “edit/compile/render/view cycle”. Don’t know which system you are referring to, but I can see many platforms that instantly render, and you can view “source” and rendered side by side. The RTF was a mess with people leaving their fonts and font sizes all across the place. A semantic doc structure is much better to maintain a consistent documentation. This doesn’t impede that you can format personally your documents like you wish.
I think the way to go for “literate programming” is a mixed markdown / code model like Jupyter notebooks.
BTW I agree that String and SinOsc should not use the same template for documentation and belong to entirely different perspectives.
1 Like
I guess a rich text editor could store to markdown, but I’m not sure the Qt one will do that.
Having edit and render windows side by side is still a kind of mode switch.
You wouldn’t have two views if you were just reading.
The idea of a rich text editor is that you edit inside the rendered view.
The rendered text around the edit cursor maintains it’s rendering as you type.
It’s true, I don’t know of a markdown editor that does this!
If SC decided it wanted a rich text editor but didn’t want to have different fonts, perhaps it could hide the font selector?
Since the Qt rich text editor is writing to Html I imagine it might allow structural editing, H1, H2, Em, Ul, Ol, Li &etc.
Why not have documentation written in markdown but parsed to HTML and rendered in the web view? That is, in my opinion, a trivial thing to solve technically.
Cue the old debate between WYSIWYG and WYSIWYM… (I’m firmly in the second camp tbh – Writing? org-mode + LaTeX > MS Word. Scoring? LilyPond > Sibelius… etc.).
hjh
Yes, and this is connected to a basic tension in SC, i.e. “who is SC directed at”?
The 1996 ICMC paper says SC began as “a Max object called Pyrite”.
https://quod.lib.umich.edu/i/icmc/bbp2372.1996.078/
So in a concrete sense SC was first directed at people who used Max.
That is, music and sound design (and perhaps signal processing) people.
Perhaps, over the years, SC has shifted focus a bit.
Perhaps the edit views of the old and new SinOsc help files reflect this.
Perhaps which you prefer depends on background and so on.
Still, if people are thinking of working on the help system again, then I think it’s worth thinking about these basic questions again as well.
Ps. In case anyone’s interested below is a picture of the “edit view” of the old SinOsc help file. (I don’t have an old SC so this is just the file opened in gmail, but you can get the idea.) For people who haven’t edited any of the new help files, you can get an “edit view” by following the link at the bottom of the SinOsc page in the scide help browser. Each has virtues, but they’re very different ones!
