Additively pass Quarks to supercollider (both `sclang` and `scide`) from the command line?

Questions
#1

Hi folks!

Is there a way to additively provide Quarks to the command line invocations of sclang and scide? Perhaps environment variables or command line arguments that can be used for this?

Motivation

I’m attempting to write a Nix flake for declaratively configuring reproducible supercollider environments that not only provide supercollider with a desired set of plugins (nixpkgs already handles this), but a desired set of Quarks in a ready-to-go manner too.

In order to achieve a fully declarative approach, I’d like to avoid mutating the user’s existing sclang_conf.yaml configuration and instead provide them directly to the running supercollider instance somehow, whether that be environment variable, command line argument, or applying some patch to supercollider itself (less ideal).

Existing (lacking) approach:

So far for sclang I’ve been using the -l flag to pass a custom sclang_conf.yaml configuration that declares the location of the provided Quarks. However, this is not ideal for a couple of reasons:

  1. It overrides the user’s existing sclang_conf.yaml configuration. I’d prefer not to do this in order to allow the user to provide Quarks using supercollider’s traditional approach too if they wish.
  2. The flag is not available for scide.

Any tips or tricks on how to go about additively providing Quarks to both sclang or scide invocations would be greatly appreciated!

#2

The IDE itself is responsible for choosing an sclang_conf.yaml, so it feels a bit weird to select one as a command line argument. I believe this is stored in the preferences for the IDE, so it’s possible you could provide a preferences file that specifies the expected yaml - users can always repoint in the IDE of course.

I think scang_conf.yaml is really the most correct way to specify a reproducible configuration for sclang. If we wanted the ability to have cascading quark configurations (e.g. an “environment” one, and a user one on top of that), one way to start would be to simply allow multiple yaml’s and merge them when we load. This would be straightforward to implement via allowing multiple -l flags, but probably what we’d want is an additional environment variable to specify an “environment” yaml file - something like SCLANG_ENVIRONMENT_YAML=/path/to/sclang_config.yaml. This would play better with the various SuperCollider IDE’s and editing environments.

#3

Agreed, having the ability to merge sclang configurations sounds nice in theory.

Taking a closer look however, it’s not clear how a merge operation might behave without any sense of order/priority. E.g. what do we do in the case that one configuration specifies an exclude directory, but another specifies the same directory as an include directory?

Proposal: SCLANG_CONF_PATH

One solution that might help to clarify merge behaviour and offer more flexibility to users could be to accept a SCLANG_CONF_PATH environment variable.

Conflicts between include and exclude paths could be resolved by having the order of priority follow the order of the configurations specified in the SCLANG_CONF_PATH. The non-conflicting content of each config listed in the path would be merged.

Example

In the following:

SCLANG_CONF_PATH=/foo/sclang_conf.yaml:/bar/sclang_conf.yaml

Priority is given to /foo/sclang_conf.yaml when resolving conflicts, and the rest is merged.

Supporting Existing Behaviour

The existing behaviour in SC_LanguageConfig.cpp is:

  1. Check for config file passed via CLI.
  2. If it doesn’t exist, check user config.
  3. If it doesn’t exist, check “global” config at /etc.

Solution 1

If we were to support a SCLANG_CONF_PATH, we could get close to this behaviour by prepending the CLI argument configuration and appending the user and global configurations SCLANG_CONF_PATH. E.g. if the user passes:

SCLANG_CONF_PATH=/foo/sclang_conf.yaml

Internally, we could resolve the path as:

SCLANG_CONF_PATH=<cli-arg-conf>:/foo/sclang_conf.yaml:<user-conf>:<global-conf>

That said, this would technically be a breaking change, as the user and global configurations would be merged, whereas currently the user config overrides the global one.

Solution 2

A more accurate, non-breaking approach would be to update the behaviour to the following:

  1. Check SCLANG_CONF_YAML and retrieve the list of config paths (possibly empty).
  2. Check for config file passed via CLI.
    • If it exists, prepend it to the list.
    • Merge the list to produce final config.
  3. If CLI arg didn’t exist, check for user config.
    • If it exists, append it to the list.
    • Merge the list to produce final config.
  4. If user config didn’t exist, check for “global” config at /etc.
    • If it exists, append it to the list.
    • Merge list to produce final config.

@scztt thoughts? If 2 sounds OK, I might have a go at doing up a PR.

#4

I made a patch for SuperCollider to pick up the configuration file defined in SCLANG_CONF_YAML (if it exists). It does not yet merge with the user configuration, but I think it is a good starting point
EDIT: I managed to make it load the environment config first, then merge it with the user configuration:

The way I managed to set the includePaths might also be of interest. Quark derivations write their corresponding extension path, as well as the extension paths of their dependencies(which I pass as propagatedBuildInputs) into a file inside the nix-support.

Then, when generating the language configuration file, each Quark will have its path written inside the includePaths field.

This allows for the dependencies to be propagated without having to copy or link them to another place, avoiding the issue of duplicates.

This functionality seems to work just fine alongside other tooling such as scide, and fits well with my usage, but I am still open to suggestions.