Tree-sitter support for SuperCollider

Hello everyone
I am happy to announce a little pet project: I have been mapping the sclang grammar to be used with the awesome tree-sitter code parser. Tree-sitter allows for advanced syntax highlighting and very precise and scope aware code analysis. For now it is mostly used to create very nice code highlighting but because it is a very precise node tree of the document it could potentially have more usecases (such as very precise syntax errors that are generated as you type). One of the things that I like the most about tree-sitter is that it reparses each node as you type (and not the whole document) which gives you immediate visual information about the code you are typing.

The superCollider grammar for tree-sitter is here:

And more info on tree-sitter:
tree-sitter.github.io/

4 Likes

The grammar has been added to neovim via the nvim-tree-sitter plugin. To use it just open neovim and type :TSInstall supercollider.

If anyone feels like adding support for this grammar to other code editors, then please do so! I only use neovim so that’s one I have added support for. Would be awesome to see this in VSCode, Atom etc.

2 Likes

Here is an example of what the node tree looks like under the hood

(
	{|freq=110|
		var sig = SinOsc.ar(freq: freq);
	
		sig * 0.5
	}.play
)

Is parsed as:

(source_file [0, 0] - [7, 0]
  (code_block [0, 0] - [6, 1]
    (function_call [1, 1] - [5, 7]
      (receiver [1, 1] - [5, 2]
        (function_block [1, 1] - [5, 2]
          (parameter_list [1, 2] - [1, 12]
            (argument [1, 3] - [1, 11]
              name: (identifier [1, 3] - [1, 7])
              value: (literal [1, 8] - [1, 11]
                (number [1, 8] - [1, 11]
                  (integer [1, 8] - [1, 11])))))
          (variable_definition [2, 2] - [2, 33]
            name: (variable [2, 2] - [2, 9]
              (local_var [2, 2] - [2, 9]
                name: (identifier [2, 6] - [2, 9])))
            value: (function_call [2, 12] - [2, 33]
              (class [2, 12] - [2, 18])
              (class_method_call [2, 18] - [2, 21]
                name: (class_method_name [2, 19] - [2, 21]))
              (class_method_call [2, 21] - [2, 33]
                (parameter_call_list [2, 22] - [2, 32]
                  (argument_calls [2, 22] - [2, 32]
                    (named_argument [2, 22] - [2, 32]
                      name: (identifier [2, 22] - [2, 26])
                      name: (variable [2, 28] - [2, 32]
                        (local_var [2, 28] - [2, 32]
                          name: (identifier [2, 28] - [2, 32])))))))))
          (binary_expression [4, 2] - [4, 11]
            left: (variable [4, 2] - [4, 5]
              (local_var [4, 2] - [4, 5]
                name: (identifier [4, 2] - [4, 5])))
            right: (literal [4, 8] - [4, 11]
              (number [4, 8] - [4, 11]
                (float [4, 8] - [4, 11]))))))
      (instance_method_call [5, 2] - [5, 7]
        name: (method_name [5, 3] - [5, 7])))))

This is phenomenal mads !!!

One immediate use-case for this would also be to fix the SuperCollider IDE’s long-broken auto-indent logic. I’ve tried several times to improve this, but it’s a pretty tough problem if you don’t have a proper AST and I can never get all of the cases right.

There’s another interesting use case for this as well… sclang would gain a lot of power if it could introspect into it’s own AST. Currently, our AST’s are represented as C++ objects, which - for various reasons - are difficult to “keep around” after compilation, and to expose to sclang. But, a much simpler solution would be to have our C++ AST simply assemble a tree-sitter AST. We’d be skipping the most interesting part of TS - the parsing - and just assembling the nodes directly (we’d need to use some private interfaces for this, but it looks do-able). The advantage here is that sclang would gain the ability to both inspect it’s own AST and do the cool “live” parsing that TS provides, which would be quite interesting for e.g. creating livecoding languages.

VSCode has good tree-sitter support, I may try to add this at some point…

That’s great Scott!
I should say that the grammar I have written is still work-in-progress (see the readme for a todo list) but I would say it has 95% of the sclang grammar implemented now (and I am constantly new esoteric language features (mainly by stress testing the tree sitter parser with Fredrik Olofsson’s tweets)).

I think that all sounds like a great idea. I think it would be awesome to implement it in the scide (and one of the dreams here is to allow beginners to see syntax errors precisely and immediately without having to wait until executing the code to rummage through the post window for details)*.

If anyone is new to tree-sitter then this talk is a pretty good intro

*The grammar needs to be fully implemented for this to work properly of course

Nice :sunglasses:

This seems relevant to one of my long-standing wishlist items: Support in the IDE for custom syntax.

We allow users to create custom syntax using the interpreter preprocessor.

The IDE rigorously enforces one and only one syntax. Particularly restrictive to me is that the IDE permits running code interactively only for .sc and .scd files (execution is disabled for other file extensions :anguished: ), and these file types handle syntax highlighting in one and only one way. Implicitly, then, the IDE discourages live coding dialects. I’m quite sure this is not the message we want to send, but it is the reality now. My live coding dialect looks hideous in the IDE, and there is currently nothing I can do about it except switch to emacs.

If tree-sitter parsing were integrated into the IDE, it would open the door to custom tree-sitter profiles for other file extensions.

It would also be an opportunity to address some of the nagging inconsistencies in cursor navigation in the IDE. Some of them are quite silly indeed.

hjh