If we assume that wake-ups will be scheduled on a non-audio clock, then we’re already in a problem space where we need to manage drift (this is pretty normal for audio applications)
Not necessarily. The client can actually be driven by the Server. This doesn’t have to happen synchronously. Instead the server can notify the client whenever it has computed a new block of audio. The client scheduler wakes up, increments its logical time and dispatches any scheduled functions/commands that fall in this time slice. The client might either run in a dedicated thread or process and simply waits on a (named) semaphore. Alternatively, the server might send a message over a (local) network connection, e.g. for remote servers. (In fact, a single remote server could drive several clients.)
Consequently, the server simply dispatches incoming OSC bundles based on logical sample time.
Of course, the logical sample time will drift from the system time. This is unavoidable. Personally, I hardly ever feel the need to work with system time, except maybe for long-running installations. Pure Data, for example, only works with logical sample time; you simply can’t schedule things precisely in terms of system time, but in turn you get deterministic sub-sample accuracy. If I want a section to be 60 seconds, I really want it to take 2880000 samples @ 48 kHz, not 60 seconds as measure with an atomic clock.
Personally, I would not even try to mix logical sample and system time. If you use the former, forget about the latter.
A more tricky case is controlling several servers from a single client. If all servers are local, the individual sample clocks won’t drift apart - at least if the servers are driven by the same audio device. But if you control several remote servers, then the individual sample clocks will drift. In that case, it might make more sense to stick with a single system time clock.
Another tricky thing is how to dispatch incoming OSC bundles relative to a sample clock. You can use a time DLL filter to compensate for drift - as scsynth is currently doing -, but you somehow have to tell the OSC receiver to use the appropriate clock source. This is largely a matter of API design. E.g. OSCFunc could have an (optional) clock argument.
Generally, much of SuperCollider’s complexity wouldn’t exist if a client could only ever control a single server… In that case, the whole client scheduler could simply be driven - asynchronously(!) - by the audio callback.