SuperCollider VSCode / Language Server Protocol support

scztt · May 24, 2023, 6:10pm

I’ve made a more usable and formal release of the VSCode plugin and Language Server. While it’s still alpha quality, I’ve been using it for larger projects and it’s working quite well. I would be super happy to get feedback and bug reports so I can clean up some of the behavior.

Setup instructions are here:

Please note especially that it requires a develop build of SuperCollider (download links are in the instructions), though if you’re already running develop locally you’ll be fine. It doesn’t affect your current SuperCollider installation at all, apart from installing the LanguageServer quark and dependencies.

UPDATES

Restarting sclang from inside VSCode works much better (see “SuperCollider: Restart sclang” command)
Documentation system is working (see “SuperCollider: Search Help”) command. It works best if you run SCDoc.renderAll() once, so all of the doc files are pre-rendered.

Methods and constructors now show full argument hints. If you’re in the middle of typing out a method call and they go away, you can always show them with the “Trigger Parameter Hints” command
Lots of minor improvements to autocomplete.
VSCode sees code blocks as regions. If you start your code block with a comment, e.g. ( // SOMETHING then VSCode will show SOMETHING in the breadcrumb bar when navigating. This allows you to easily navigate between code blocks with e.g. keyboard shortcuts or UI.

scztt · May 24, 2023, 8:18pm

There’s the new UDP port changes from December that barely missed the last release, and some help file things that are sort of required for the help system to work.

Sam_Pluta · May 25, 2023, 10:29am

So excited for this Scott. I’m going to check it out this week. Thanks for your work on it.

Sam

scztt · May 25, 2023, 11:55am

Some things that would be very helpful if anyone tries the plugin:

Can you post the location of your sclang executable if you’re on Win / Linux? There’s a pretty “normal” install location on Mac, but there are some variations on Windows and definitely on Linux. I’d like to improve the defaults so the extension can find common SuperCollider install locations without needing to specify them.
How does it work with the way you organize projects? I keep related .scd files together in a single folder, and open that folder in VSCode as a workspace - this workflow works well. But there may be other styles of working that would be useful to support.
The SuperCollider documentation SHOULD be re-colored to match your VSCode theme. If you see colors in the documentation that are ugly or make it un-readable, can you post a screenshot and the theme you’re using?
There is syntax highlighting for both .sc/.scd files and for the post window (e.g. errors etc). These are works-in-progress - if you see instances where the highlighting is blatantly wrong, screenshots and code snippets would be helpful (especially if you can experiment a little to find what’s causing the problem).
The way syntax highlighting combines with color themes is complex. If you have a theme and the highlighting doesn’t look good or seems missing, let me know the theme and I can look into it.

smoge · May 30, 2023, 7:56pm

I made a first attempt using LSP in vscode (Fedora 38 OS). The basics work, sclang starts, syntax recognized, server pops up in pipwire-jack, etc. But I didn’t see any LSP server functionalities (format code, code actions, etc.). Some error messages I got:

CALL STACK:
DoesNotUnderstandError:reportError
arg this =
< FunctionDef in Method LSPConnection:prHandleMessage >
arg error =
< FunctionDef in Method Deferred:then > (no arguments or variables)
Routine:prStart
arg this =
arg inval = 17.463464387
^^ ERROR: Message ‘isOpen’ not understood.
RECEIVER: nil

[Error - 3:52:08 PM] Request textDocument/formatting failed.
Message: DoesNotUnderstandError
Code: 691960848

scztt · May 30, 2023, 8:18pm

I pushed something that should fix. If you update the LanguageServer quark (“Update LanguageServer.quark” in the command palette, or just do it via the quarks system directly) and restart it should be resolved. If you still see problems / no language server functionality after updating, try changing the “Supercollider: Language Server Log Level” setting to “info” and restarting (in json, "supercollider.languageServerLogLevel": "info") - this will show a detailed log, hopefully it will help identify what’s going on.

Thanks for the report / triaging. I haven’t been able to test on linux and I think the configuration details are the most complex there so it’s very helpful to have someone trying it out. FWIW this particular problem occurred because I don’t have the code formatter is not enabled by default in the VSCode plugin because it requires a few more complex setup things and is relatively untested - I think i did local testing with the code formatter enabled, so I didn’t notice this problem.

smoge · May 30, 2023, 8:27pm

Thank you for the quick reply. I’m using upfated quark, and I compiled your supercollider branch (should I use the ‘official’ dev branch at this point?)

I followed your instructions, still got some errors:

a FunctionDef	0x3fd1e40
	sourceCode = "<an open Function>"
Function:prTry	0x2fbad80
	arg this = a Function
	var result = nil
	var thread = a Routine
	var next = nil
	var wasInProtectedFunc = true

CALL STACK:
DoesNotUnderstandError:reportError
arg this =
< FunctionDef in Method LSPConnection:prHandleMessage >
arg error =
< FunctionDef in Method Deferred:then > (no arguments or variables)
Routine:prStart
arg this =
arg inval = 16.817976148
^^ ERROR: Message ‘putString’ not understood.
RECEIVER: nil

[LANGUAGESERVER.QUARK] Responding with: Content-Length: 76

{“error”: {“code”: 1560068730,“message”: “DoesNotUnderstandError”},“id”: 8}

[Error - 4:24:56 PM] Request textDocument/formatting failed.
Message: DoesNotUnderstandError
Code: 1560068730
[LANGUAGESERVER.QUARK] Message received: 20.237589886, a NetAddr(127.0.0.1, 46232), Content-Length: 130

{“jsonrpc”:“2.0”,“id”:9,“method”:“textDocument/codeLens”,“params”:{“textDocument”:{“uri”:“file:///home/bbarros/impro/Main.scd”}}}

[LANGUAGESERVER.QUARK] Expecting 130 bytes, received 130 so far
[LANGUAGESERVER.QUARK] Found method provider: a CodeLensProvider
[LANGUAGESERVER.QUARK] Responding with: Content-Length: 286

{“id”: 9,“result”: [{“range”: {“start”: {“character”: 0,“line”: 22},“end”: {“character”: 1,“line”: 268}},“command”: {“title”: “ Evaluate block”,“command”: “supercollider.evaluateSelection”,“arguments”: [{“start”: {“character”: 0,“line”: 22},“end”: {“character”: 1,“line”: 268}}]}}]}

smoge · May 30, 2023, 8:51pm

Another thing I noticed (I don’t know if it’s already implemented): nothing appears in the “outline” tab when I open a class file, as it does with a python or haskell file.

scztt · May 30, 2023, 9:45pm

I also just pushed a fix for the error you posted (also a CodeFormatter issue). Hopefully an update will resolve.

The outline tab will currently show only code blocks (e.g. sections surrounded by ()'s in .scd files) - if you put a comment after your opening (, this text will be shown in the outline. Showing a full structured code outline requires more sophisticated parsing - the parsing part is finished (via @lnihlen’s sparkle parser), but I haven’t yet hooked it up to the language server, but this is the next major item I’ll work on.

(Also, once you have things working, you can set the logging preference back to “warning” or “error” to avoid so many log messages)

smoge · May 30, 2023, 10:06pm

Now vscode just pops up a window. No errors.

On “outlines”: yes, that will be very useful when implemented, specially for long .sc textfiles.

The project looks very promising, I’m looking forward to new updates. It’s already useful for basic usage, and the possibilities are very exciting.

scztt · May 30, 2023, 10:38pm

Keep me posted about how it’s working for you. I think the formatter message you’re seeing is harmless, but I should be able to hook the formatter up soon, which should resolve regardless.

smoge · May 30, 2023, 10:42pm

Is it the right place for bug reports and bug fixes?

Also, it would be just fantastic to see a “lsp-supercollider” in emacs hooking it up with lsp-mode. I guess it’s part of the big plan.

scztt · May 30, 2023, 10:55pm

If you’re interested in hacking on it a bit, adding basic support for .sc files is really simple. The class that handles this is DocumentSymbolProvider - the response is constructed in :onReceived. There’s a branch for sc versus scd files, and the sc is not implemented, but the implemention here would look like:

Iterate through LSPDatabase.allMethods
Make an IdentityDictionary mapping file paths from Method:filenameSymbol to the list of methods in that file, e.g. LSPDatabase.allMethods.do { |method| filesToMethods[method.filenameSymbol] = filesToMethods[method.filenameSymbol].add(method) }. This should be done in a method on LSPDatabase, e.g. LSPDatabase:methodsForFile { |path| ... }, so you can cache this map once it’s constructed (similar to allMethods).
For the requested file, get all the methods via the new methodsForFile, and get the locations of each via the existing LSPDatabase:renderMethodRange. This could be optimized to cache the results and to get multiple methods at once, rather than re-reading the file for each one.
Construct the response like the existing one in DocumentSymbolProvider.sc:31-34, where the name is the method name, and the range and selectionRange should be the result of LSPDatabase:renderMethodRange
I believe this won’t work correctly for files with multiple classes defined in the same file (at least not without work…) due to a bug/idiosyncrasy in the way sclang stores file positions.

Doing this should provide a nice outline for sc files as well. I was going to skip this and work on the “full parse” method, but if you got the above working I would definitely add it since the “proper” solution may take a while.

scztt · May 30, 2023, 10:56pm

Go ahead and report them here GitHub - scztt/vscode-supercollider or here GitHub - scztt/LanguageServer.quark. The former is more for vscode specific usability things, the latter is more for (mis-)behavior of the language server - but if you’re not so important, it doesn’t really matter where you file them.

scztt · May 30, 2023, 10:58pm

I’d love to support this - vim as well! The current blocker is that I’m doing communication via a UDP port, which I believe takes some extra work to support in vim/emacs. This limitation will be removed eventually but this requires some small but tricky sclang work. If someone did the UDP connection work in vim or emacs, then I think everything would work fine right now.

smoge · May 30, 2023, 11:11pm

If I’m not mistaken, you can use UDP in emacs (osc.el) creating a datagram server:

" Emacs Lisp programs can open stream (TCP) and datagram (UDP) network connections"

smoge · May 31, 2023, 3:09am

Perhaps an interesting idea: some special kind of markup in any comment line that would create a “tag”, just like you did with code blocks in parentheses showing up in the “outline” tab.

scztt · May 31, 2023, 3:20am

I need to do a little more refactoring to get things there, but my thinking is that many of these LSP features would be extensible so that e.g. you could provide additional custom outline features, autocompletions, etc. The most basic thing is: I have complex synths and patterns that I can’t always remember the parameters of, and I’d like to provide custom auto-completion for them. I’d like people to be able to make quarks that integrate directly with the language server for things like livecoding. I think tags would be cool - probably the kind of thing that would make a good extension rather than a “core” language server feature.

smoge · May 31, 2023, 3:35am

Got it.

Another thing concerning all this would be the idea of a “project” or “workspace”, where one could have access to symbols not just in one text file. I know that people have different approaches to using SC, so that would be another good extension idea for the LSP.

scztt · May 31, 2023, 3:44am

Right now, the language server has access to:

all symbols (classes/methods) in the class library
all environment variables in the current environment (only for autocomplete)
all names for singleton / def style objects like Ndef, Pdef, OSCdef etc. (only for autocomplete)

None of these things really needs a concept of a “workspace” (though VSCode has workspaces and some LSP features leverage this). Some kind of knowledge of e.g. local variables and scope would be good, but that feels very far down the priority list - though if you have specific idea about how this would work, I’d love to hear them!