SuperCollider VSCode / Language Server Protocol support

I’ve been slowly working on proper + full-featured Visual Studio Code support for SuperCollider / sclang for a little while now. The core functionality can be implemented mainly in Language Server Protocol, an open standard that specifies how an IDE can connect to a language runtime like sclang. Most or all of the existing SC IDE functionality can more or less be expressed in terms of LSP, along with a pile of interesting new functionality.

The advantage here is that LSP is an open standard, implemented by many code editors, which means in theory proper LSP support allows a rich editing for a variety of tools. In the real world, support for a protocol does not always mean perfect functioning, but a proper LSP implementation could give us an ScIDE-level editing experience in VSCode, Atom, Emacs, neovim, Sublime, and many others.

The potential scale of this project is very large - there are a million details to get right! - but I think the benefit is also very large. I’d like to open the development effort up to the community, so we can build something together that represents the next step in how we program and perform with SuperCollider.


There are many entry-points for helping out with this project, even if you’re not a programming wizard. Some specific areas of contribution that would be very valuable in moving this forward.

  • People willing to brave fiddly configuration and error messages to try out vscode support and patiently report problems.
  • Devotees of another IDE that supports LSP, that are willing to work on supporting this in their favorite editor.
  • LOTS of sclang programming to implement / solidify pieces of the language server (most of it is written in sclang)
  • JavaScript / Typescript programmers who can help me improve the VSCode extension (I’m a newb at TS and my code is trash…)
  • More extensive unit tests for code completion features (these tests are simple and written in sclang)
  • C++ programmers who can do some basic detail-oriented network communication work in sclang (no deep experience of the SC codebase required for this)
  • Parts of the SC documentation renderer need to be reworked for documentation to properly work in VSCode
  • SCIDE code formatter needs to be improved and broken-out to a standalone library/tool.
  • Implementing new parts of the LSP protocol, with a careful and detailed-oriented eye to usability / user experience.

I’m tracking open tasks on the project page. Tasks are labelled with the area of expertise (e.g. [c++]), lightweight tasks are marked with [easy].

Here are the pieces of the project:

GitHub project page
SuperCollider LSP branch
vscode-supercollider
LanguageServer.quark

How to get started:

If you have technical issues or feature requests, please use the GitHub issues page so we can leave this topic free for general discussion.

9 Likes

Woohooo. This is so exciting. You are an absolute hero. SC LSP is a long time dream. I will contribute as much as I can when I am back to health! Especially on the neovim side.

1 Like

Yeah, this is wonderful; really happy to see it underway!

I’m buried under other projects for the moment, but once things open up a bit, I’ll look forward to checking in on that list of tasks and seeing if there’s anything I can do to contribute.

To get it working in Neovim we need to add a config to nvim-lspconfig (GitHub - neovim/nvim-lspconfig: Quickstart configurations for the Nvim LSP client).

Here’s an example for pyright (python LSP):

Fantastic @scztt <3 veeery exciting indeed. YES!
@madskjeldgaard lets add it :slight_smile: → nvim lsp client hackathån in the weekend.

Great to hear this! I once also started to implement this but found myself in the difficult situation that the lack of a proper network stack in sclang limits the communication to OSC and therefore a translation layer would be necessary which I found difficult to reason about and abandoned it.

On the other hand I also disliked the idea of providing to a project which more and more aims for monpolistic stand in the editor domain, although a LSP is not bound to VSCode.
One should definitely support GitHub - VSCodium/vscodium: binary releases of VS Code without MS branding/telemetry/licensing as well
Also: what do you consider the impact of this to the ScIDE as some really awesome features would probably only be available in vscode [at least in the beginning?] - would the IDE maybe get abandoned?

I would love to try out and use the language server for another IDE which I support/develop, GitHub - capital-G/sc_kernel: SuperCollider Kernel for Jupyter

I share your concerns - I chose to support LSP for reasons that are probably obvious - it’s the only protocol like this that’s widely used. I chose to put my effort into VSCode specifically because it’s the most likely candidate to provide a critical mass of features to get people to actually use it, and represents one of the more full-featured LSP implementations without requiring extra work (and, trusting that Mads and company would be working on neovim support in parallel).

There are subtle usability things about ScIDE that VSCode is unlikely ever to surpass, because it was built for SC. Other editors will quickly blow past it in terms of raw feature set, but I’m sure the ScIDE will continue to be valuable to a subset of people who are attached to specific details of it’s UX. At the moment I can’t really use VSCode for serious work because 10+ years of muscle memory with ScIDE is too much at this early stage of the project :slight_smile: - so in that sense ScIDE is unlikely to be abandoned any time soon by users.

But from a development perspective, ScIDE is already basically abandoned - no one is adding features or contributing large fixes to it, and it’s built on a stack that a vanishingly small number of people have expertise in (the “old” QT widget framework and C++). When QT finally deprecates the c++ widgets, there’s no good path to keeping it alive. There aren’t really resources to maintain a full IDE stack in the SC dev community anyway, and any extra c++ skills hanging around would be better put towards working on SC itself rather than e.g. reproducing features that 40 other editors already have.

I don’t mean to devalue the IDE itself - I’ve been combing through its features and there are pieces of it’s UX that are very well designed and will be a lot of work to reproduce in VSCode or other editors! But I want git Integration and code folding and snippets and error messages with source code links, and I’d rather let the dev community of VSC and neovim write those features so I can make music :slight_smile:

3 Likes

One that I would miss is the left-paren “choose the responding class” dialog. If that can’t be replicated, it would be pretty much a deal-breaker for me (though I do get the point about eventually vanishing support for our IDE).

One thing that might be nice is to allow user defined syntax, perhaps based on file extension, or maybe a tag at the top of the file. The preprocessor supports custom syntax, but the IDE strictly follows sclang syntax – some cognitive dissonance there.

hjh

LSP is huge in Neovim land also, just FYI :slight_smile:

1 Like

This is mostly working now via LSP.


There are subtle usability things to fix - aligning the class names would be nice for readability, and using * denotation for class methods rather than showing Meta_BlahBlah is probably easier to read as well. They’re alpha sorted, but for some reason Meta classes are shown first. FWIW

For syntax highlighting, multiple embedded syntaxes are allowed via TextMate grammars (which is what is being used for highlighting - for now…). But writing these is hard, so I don’t expect this will become SO common? The LSP server is written in sclang and extensible, so a quark can define it’s own custom completion handlers (and in the future, things like hover text and code actions) - this could potentially make “mini” custom languages like ixi quite cool.

But isn’t the selection of the appropriate method only an intermediate step as the LSP could tell you which methods are available for the object while writing the code? This is something I would really love to see.

It does this for cases where it knows (or can make a reasonable guess…) about what the type of the object is: ~environmentVariables, and ClassNames. In the example case, there are no guarantees about what type something is - even inferring the type in cases like this is almost impossible, since SC provides no guarantees that in a case as obvious as: something = MyObject(); something.method(), that something is a MyObject (MyObject:new is just a method and can return whatever it wants…).

That doesn’t mean we shouldn’t TRY to give reasonable guesses as to the method: but this may not ever be THAT accurate, and anyway will take a lot of effort to get working at all.

1 Like

I wasn’t really providing completions in the correct format - here’s what it looks like now that I’m making better use of the LSP API:

2 Likes

Pseudo-UGens are a solid example of this.

hjh

I just want to chime it to say how 110% in support of this I am. I started using VSCode last year for another project and I loved it. Going forward this seems like the obvious way to be able to maintain SC and push it into the future.

Sam

1 Like

This might be rather out of scope, but Hernan Wilkinson’s “Live Typing” work in Cuis/Smalltalk is very nice!

The implementation mechanism is quite simple.

There’s a “California Smalltalkers” talk at https://www.slideshare.net/hernanwilkinson/live-typing-california-smalltalkers.

(Implementation notes at slide 21, performance measurements at slide 43, there’s a link to the recording.)

Type collection seems to slow things down by about 1.5 times and grow the image size by about the same amount.

Neovim attempt

(should this be a seperate thread?)

I did a quick attempt at setting this up in neovim today but I didn’t quite succeed. Sharing my notes here in case someone else would like to pick up from here and make a more serious attempt at it.

Firstly, I installed the network refactor branch of SuperCollider as described by Scott along with the quark.

I forked (locally) neovim-lsp-config and added a server config called supercollider.lua in nvim-lspconfig/lua/lspconfig/server_configurations at master · neovim/nvim-lspconfig · GitHub looking like this:

local util = require 'lspconfig.util'

local bin_name = 'sclang'
local cmd = { bin_name, '-i vscode' }

-- local lspreadport = 58110
-- local lspwriteport = 58111

return {
  default_config = {
    cmd = cmd,
    filetypes = { 'supercollider' },
    root_dir = util.find_git_ancestor,
    single_file_support = true,
    settings = {
	-- TODO
      supercollider = { },
    },
    -- capabilities = util.default_config.capabilities or vim.lsp.protocol.make_client_capabilities(),
  },
  commands = {},
  docs = {
    description = [[
https://github.com/scztt/vscode-supercollider

`vscode-supercollider` is a lsp server for SuperCollider
]],
  },
}

And then added this to my setup function for nvim-lspconfig:

-- SuperCollider
require'lspconfig'.supercollider.setup{}

When opening a supercollider file, it detects the server but does not attach it. Not sure why. Here’s the output of :LspInfo

 Language client log: /home/mads/.cache/nvim/lsp.log
 Detected filetype:   supercollider
 
 0 client(s) attached to this buffer: 
 
 Other clients that match the filetype: supercollider
 
 Config: supercollider
 	filetypes:         supercollider
 	root directory:    /home/mads/.local/share/SuperCollider/downloaded-quarks/mk-synthlib
 	cmd:               sclang -i vscode
 	cmd is executable: true
 	autostart:         true
 	custom handlers:   
 
 Configured servers list: clangd, supercollider, psalm, cssls, bashls, cmake, vimls, html, rust_analyzer, pyright, sumneko_lua
1 Like

Oh and I could tell from htop that it does in fact spawn the sclang -i vscode process.

Ah just found this in my log which happens when opneing a supercollider file with lsp enabled

WARN  2022-03-24T18:08:18.505 550041 server_start:154: Failed to start server: address already in use: /tmp/nvimrDJwwY/0

sorry for spam

ps. I have no idea what I’m doing

Hey Mads! Here’s where we set things up to launch sclang from vscode:

Unless it’s hiding somewhere outside of what you posted, what you’re PROBABLY missing are these environment variables, which is what we are currently using to communicate between the language client and language server.

If you have the environment variables set, you should see things like this from sclang when you launch:

[LSPCONNECTION] Starting language server, inPort: 58111 outPort:58110
[LSPCONNECTION] Adding provider for method 'initialize'
***LSP READY***
[LSPCONNECTION] Message received: 4.56623792, a NetAddr(127.0.0.1, 58110), {"jsonrpc":"2.0","id":0,"method":"initialize","params":{"processId":76355,"clientInfo":{"name":"Visual Studio Code","version":"1.65.2"},"locale":"en-us","rootPath":null,"rootUri":null,"capabilities":{"workspace":{"applyEdit":true,"workspaceEdit":{"documentChanges":true,"resourceOperations":["create","rename","delete"],"failureHandling":"textOnlyTransactional","normalizesLineEndings":true,"changeAnnotationSupport":{"groupsOnLabel":true}},"didChangeConfiguration":{"dynamicRegistration":true},"didChangeWatchedFiles":{"dynamicRegistration":true},"symbol":{"dynamicRegistration":true,"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"tagSupport":{"valueSet":[1]}},"codeLens":{"refreshSupport":true},"executeCommand":{"dynamicRegistration":true},"configuration":true,"workspaceFolders":true,"semanticTokens":{"refreshSupport":true},"fileOperations":{"dynamicRegistration":true,"didCreate":true,"didRename":true,"didDelete":true,"willCreate":true,"willRename":true,"willDelete":true}},"textDocument":{"publishDiagnostics":{"relatedInformation":true,"versionSupport":false,"tagSupport":{"valueSet":[1,2]},"codeDescriptionSupport":true,"dataSupport":true},"synchronization":{"dynamicRegistration":true,"willSave":true,"willSaveWaitUntil":true,"didSave":true},"completion":{"dynamicRegistration":true,"contextSupport":true,"completionItem":{"snippetSupport":true,"commitCharactersSupport":true,"documentationFormat":["markdown","plaintext"],"deprecatedSupport":true,"preselectSupport":true,"tagSupport":{"valueSet":[1]},"insertReplaceSupport":true,"resolveSupport":{"properties":["documentation","detail","additionalTextEdits"]},"insertTextModeSupport":{"valueSet":[1,2]}},"completionItemKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25]}},"hover":{"dynamicRegistration":true,"contentFormat":["markdown","plaintext"]},"signatureHelp":{"dynamicRegistration":true,"signatureInformation":{"documentationFormat":["markdown","plaintext"],"parameterInformation":{"labelOffsetSupport":true},"activeParameterSupport":true},"contextSupport":true},"definition":{"dynamicRegistration":true,"linkSupport":true},"references":{"dynamicRegistration":true},"documentHighlight":{"dynamicRegistration":true},"documentSymbol":{"dynamicRegistration":true,"symbolKind":{"valueSet":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26]},"hierarchicalDocumentSymbolSupport":true,"tagSupport":{"valueSet":[1]},"labelSupport":true},"codeAction":{"dynamicRegistration":true,"isPreferredSupport":true,"disabledSupport":true,"dataSupport":true,"resolveSupport":{"properties":["edit"]},"codeActionLiteralSupport":{"codeActionKind":{"valueSet":["","quickfix","refactor","refactor.extract","refactor.inline","refactor.rewrite","source","source.organizeImports"]}},"honorsChangeAnnotations":false},"codeLens":{"dynamicRegistration":true},"formatting":{"dynamicRegistration":true},"rangeFormatting":{"dynamicRegistration":true},"onTypeFormatting":{"dynamicRegistration":true},"rename":{"dynamicRegistration":true,"prepareSupport":true,"prepareSupportDefaultBehavior":1,"honorsChangeAnnotations":true},"documentLink":{"dynamicRegistration":true,"tooltipSupport":true},"typeDefinition":{"dynamicRegistration":true,"linkSupport":true},"implementation":{"dynamicRegistration":true,"linkSupport":true},"colorProvider":{"dynamicRegistration":true},"foldingRange":{"dynamicRegistration":true,"rangeLimit":5000,"lineFoldingOnly":true},"declaration":{"dynamicRegistration":true,"linkSupport":true},"selectionRange":{"dynamicRegistration":true},"callHierarchy":{"dynamicRegistration":true},"semanticTokens":{"dynamicRegistration":true,"tokenTypes":["namespace","type","class","enum","interface","struct","typeParameter","parameter","variable","property","enumMember","event","function","method","macro","keyword","modifier","comment","string","number","regexp","operator"],"tokenModifiers":["declaration","definition","readonly","static","deprecated","abstract","async","modification","documentation","defaultLibrary"],"formats":["relative"],"requests":{"range":true,"full":{"delta":true}},"multilineTokenSupport":false,"overlappingTokenSupport":false},"linkedEditingRange":{"dynamicRegistration":true},"execution":{"executeSelection":true}},"window":{"showMessage":{"messageActionItem":{"additionalPropertiesSupport":true}},"showDocument":{"support":true},"workDoneProgress":true},"general":{"regularExpressions":{"engine":"ECMAScript","version":"ES2020"},"markdown":{"parser":"marked","version":"1.1.0"}}},"trace":"off","workspaceFolders":null}}
[LSPCONNECTION] Found method provider: an InitializeProvider
[INITIALIZEPROVIDER] Found providers: [ textDocument/definition ], [ textDocument/signatureHelp ], [ textDocument/references ], [ textDocument/completion ], [ documentation/search ], [ textDocument/declaration ], [ initialize ], [ textDocument/didOpen, textDocument/didChange, textDocument/didClose ], [ textDocument/executeSelection ], [ textDocument/codeAction ], [ textDocument/codeLens ], [ textDocument/implementation ]
[INITIALIZEPROVIDER] Checking for client capability at textDocument.definition (clientCapabilities: Dictionary[ (window -> Dictionary[ (showMessage -> Dictionary[ (messageActionItem -> Dictionary[ (additionalPropertiesSupport -> true) ]) ]), (showDocument -> Dictionary[ (support -> true) ]), (workDoneProgress -> true) ]), (general -> Dictionary[ (regularExpressions -> Dictionary[ (engine -> ECMAScript), (version -> ES2020) ]), (markdown -> Dictionary[ (parser -> marked), (version -> 1.1.0) ]) ]), (textDocument -> Dictionary[ (codeLens -> Dictionary[ (dynamicRegistration -> true) ]), (formatting -> Diction...etc...)
[INITIALIZEPROVIDER] Registering provider: [ textDocument/definition ]
[INITIALIZEPROVIDER] Adding server capability at definitionProvider: (  )
[INITIALIZEPROVIDER] writing options into key definitionProvider

However, there are currently several non-standard things about the way the server is configured, so it might not work immediately with neovim even with that. I’m happy to fill in the gaps to get neovim up and running, but to he honest I’m completely at a loss figuring out what mechanism neovim uses to connect to language servers. Generally this is either a named pipe or a TCP socket (I’m using UDP sockets with VSCode, but only for dumb convenience reasons) - it’ll be a little work to connect either of those, but I just can’t for the life of me figure out what neovim expects, and afaict it’s completely missing from documentation. Probably it’s normal for language servers to just implement both mechanisms, but I don’t really want to make a random guess at which one is going to work without a way of testing. Any thoughts here?