One of my favorite aspects of SC is the super robust clocking, via SystemClock and TempoClock. I’m trying to do something similar in Python. Does anyone have any experience with this? I haven’t been able to find any pre-built equivalent that is robust and well documented. If I were to build my own, how would I go about finding out how SC’s implementation works? Any tips are appreciated, thanks!
I would look at the clocks in Supriya. I haven’t used them yet, but that is where I was going to start for the same problem.
The python async stuff really makes me appreciate how SC just handles that all for you.
Sam
Agreed on Python making you appreciate SC’s async capabilities! I’d checked out Supriya before — as far as I can tell, it’s a semi-abandoned project, and there’s no documentation of the clocking at all. I’m looking for something I can count on for the foreseeable future.
not necessarily what you are asking, but sc3nb allows for more or less seamless interaction between supercollider and (interactive?) python scripting; possibly with scheduling python scripts to sc clocks…
IIRC it is actually hard to make reliable clocks within Python and I heard that asyncio is too unreliable for this and one should instead setup something like a custom threading procedure (had a small discussion about this on a bus trip to ICLC).
I would be really interested in getting some further background on this though.
Probably it is worth to look into Foxdot → FoxDot/FoxDot/lib/TempoClock.py at master · Qirky/FoxDot · GitHub
On a related note - I ended up building my own clock on top of the Tone.js clock in Javascript. The Tone clock is super steady but behaves badly when you eg. change the tempo or sometimes if you stop and start it so I built a clock on top which does not change the underlying tempo, just how it is interpreted. Using some of the Tone.js scheduling functions I have been able to recreate a good part of the SC pattern lib in JS and get SC like behaviour.
Nice. Anything you’d be open to share here?
SC events are typically scheduled on logical time, so the underlying clock doesn’t need to be super precise.
Yes I will at some point. What I have so far is:
- A recreation of the pattern library with the most important patterns, this is pretty easy to extend. So far I have Pbind, Pseq, Pfunc, Pkey, Prand, Pxrand, Pseries, Pwhite. This part is easy to use and if you know SC you can almost use without further documentation. Like sc, you can do [.asStream … .next] or play the pattern on a clock. The syntax is slightly different, like eg.
new Pbind({
dur: new Pseq([1], Infinity),
midinote: new Pseq([60, 62, 61, 62], Infinity),
});
The pattern library has some limitation (as of now) compared to sc, for instance only certain math operations can be done to certain patterns.
- A clock built on top of the Tone.js clock. The Tone clock is never stopped and its tempo is never changed. The TempoClock on top it uses offsets and calculates in bps like sc. Tone.js needs to be running for the clock to work. Patterns can be scheduled on the clock much like sc, again slightly different syntax, but close to sc. This could probably also be used by sc users without much documentation.
The scheduling works by calculating the dur value slightly ahead of time and all the rest of the keys when the note plays. The pre-schedule window can be set a la s.latency in sc.
-
Various defPlayers which are sort of synthdefs with built in fx. Because Tone.js does not offer the equivalent of s.sync and for other performances related reasons, all voices are preallocated and some clever voice management let’s you choose how to use the voices. This in combination with a portamento settings means all patterns have PmonoArtic like behaviour and also that the individual panning of each voice can be precisely controlled to eg. create a broad chord or a fat bass note from a single pattern. Certain keys in the patterns have special meaning when used with defPlayers and there is a special defPlayer for combining several drums sounds to a drum kit. This part would need good documentation I think. Effects can be automated on an event basis from a pattern. This does not catch the very beginning of the sound but rather ramps up quickly, so a little envelope on the send if you will.
-
A ctxPlayer which lets you draw line, dots, square etc (the usual ctx stuff) with or without animation. The ctxPlayer can be played with a pattern and visual objects are thus synced to the same clock as audio events. The ctxPlayer interprets certain keys in a specific way, for instance to animate the drawing of a circle. This part also would need good documentation and I need to do some more developments to this part.
Sorry to hijack the thread with such a long post, if admins feel it should be a separate thread, feel free to move it.
trigger warning - python code approaching
i needed to make this anyway. i hardly ever do anything with clocks other than scheduling things in seconds using routines, so this pretty much does everything i need. i have tested it, but it may not be perfect. i seldom am.
it gets around python annoyances by:
- making a thread so it doesn’t block other processes
- putting asyncio coroutines in the thread. unlike threads, these can be stopped in the middle of any process.
- responding to ctr-c by killing all coroutines in the thread and emptying the routine list
- each routine can be freed on its own or all can be freed at once
- it runs forever, like in sc, by creating a new thread when you free the old one
so, this basically functions like a Routine played by a SystemClock. thats all i ever do, so that is pretty much all i need.
import signal
import asyncio
import threading
import time
class Scheduler:
def __init__(self):
self.loop = None
self.thread = None
self.running = False
self.routines = []
signal.signal(signal.SIGINT, self._signal_handler)
self.start_thread()
def _signal_handler(self, signum, frame):
"""Handle Ctrl+C signal"""
print("\nReceived Ctrl+C, stopping routines...")
self.stop_routs()
def start_thread(self):
"""Start the asyncio event loop in a separate thread"""
def run_event_loop():
self.loop = asyncio.new_event_loop()
asyncio.set_event_loop(self.loop)
self.running = True
print(f"Asyncio thread started: {threading.get_ident()}")
try:
self.loop.run_forever()
finally:
self.loop.close()
self.running = False
print(f"Asyncio thread stopped: {threading.get_ident()}")
if self.thread is not None and self.thread.is_alive():
return self.thread
self.thread = threading.Thread(target=run_event_loop, daemon=True)
self.thread.start()
# Wait for the loop to be ready
while not self.running:
time.sleep(0.01)
return self.thread
def sched(self, coro):
"""Add a coroutine to the running event loop"""
# any time a new event is scheduled, clear the routs list of finished coroutines
print(len(self.routines))
for i in range(len(self.routines)-1,-1,-1):
print(self.routines[i].done())
if self.routines[i].done():
del self.routines[i]
if not self.running or not self.loop:
raise RuntimeError("Asyncio thread is not running")
rout = asyncio.run_coroutine_threadsafe(coro, self.loop)
self.routines.append(rout)
return rout
def stop_routs(self):
"""Stop all running routines"""
for rout in self.routines:
rout.cancel()
self.routines.clear()
def get_routs(self):
"""Get all running routines"""
return self.routines
def stop_thread(self):
"""Stop the asyncio event loop and thread and start a new one"""
if self.loop and self.running:
self.loop.call_soon_threadsafe(self.loop.stop)
if self.thread:
self.thread.join(timeout=5)
self.start_thread()
# Example async functions to add to the thread
async def counter(name, wait):
"""A counter coroutine"""
i = 0
while True:
print(f"{name}: {i}")
await asyncio.sleep(wait)
i += 1
async def sched_event(name, delta):
"""A scheduled event coroutine"""
await asyncio.sleep(delta)
print(f"Scheduled event '{name}' delayed '{delta}' seconds.")
# Start the asyncio thread
scheduler = Scheduler()
# schedule an infinite routine
rout1 = scheduler.sched(counter("Counter-1", 0.5))
# schedule an event that ends
scheduler.sched(sched_event("Event-1", 5))
# get all active routines - each time a new event is scheduled, the done routines are culled, so some routines in the array may be done
scheduler.get_routs()
scheduler.stop_routs() # Stop all routines
rout1.cancel() # Cancel any individual routines
scheduler.stop_thread() # stop the entire thread. a new thread will be created and replace the old thread
@Spacechild1 could you elaborate on this?
Sure!
Take the following hypothetical example:
5.do { |i|
TempoClock.sched(i, { ~playNote.() });
}
Here we want to play 5 notes with a musical time difference of 1 beat each. Every loop iteration takes some time to process in the language. (Imagine that ~playNote is actually a pretty complex function.)
If we use the current real system time as our time base, the delta between schedules notes would be slightly different and we would not get consistent and reliable musical timing.
The current logical (system) time, however, remains constant until any function in the current call stack yields back to the scheduler (i.e. calls wait or yield). In our case, this means that the delta will be exactly 1 beat because every iteration uses the same time stamp.
This can be demonstrated easily with the following code:
(
// logical time -> prints 3x the same time
SystemClock.seconds.postln;
SystemClock.seconds.postln;
SystemClock.seconds.postln;
)
(
// real elapsed system time -> prints 3 different times
Main.elapsedTime.postln;
Main.elapsedTime.postln;
Main.elapsedTime.postln;
)
The algorithm for a scheduler with a logical time base goes something like this:
- wait until the priority queue has at least one item
- pop the top item and compare its scheduled time (T) with the current system time (S).
If T <= S, go to 5. - sleep until T
- when we wake up, sample the current system time again (S). If T > S, go back to 3.
- set the current logical time to T and run the item, then go back to 1.
As you can see, the system clock only needs to catch up with the scheduled items. The clock precision is not too important because we do not schedule on the current system time. Just like language jitter, clock jitter is tolerable to a certain degree, depending on the Server latency. This brings us to the last point.
To preserve the musical timing of our events on the Server, we typically schedule things in the future with some fixed delay (= Server latency). The main two reasons are:
-
language jitter
All code takes time to execute and certain operations may temporarily block the language thread. Scheduling Server commands into the future helps to accomodate such timing irregularities -
audio jitter
Audio is computed in blocks. Messages that arrive between control blocks will be quantized to block boundaries if they are not scheduled at least one block into the future. Note that the audio callback typically calculates multiple control blocks in a row. E.g. with a hardware buffer size of 256 samples, every callback calculates 4 control blocks of 64 samples each. This means that the Server latency should be at least as big as the hardware buffer size.
2 Likes
Cool!
The differences between SystemClock and TempoClock are:
- Internally, TempoClock retains the beat and seconds values when the tempo last changed. Then beats-to-seconds is
(beats - baseBeat) / beatsPerSec + baseSeconds. - The clock thread should be sleeping until the seconds value for the earliest item in the queue.
- Scheduling a new item may be earlier than that, so this signals the thread to wake up and check for activity.
- Changing tempo also changes the seconds value of the head of the queue, so this also signals.
Probably wouldn’t be too hard for Python folk to add that into this clock. I’m not that person, but I guess it could be done.
hjh
not perfect, but functional:
import signal
import asyncio
import threading
import time
class Scheduler:
def __init__(self):
self.loop = None
self.thread = None
self.running = False
self.routines = []
self.tempo = 60
self.wait_mult = 1
signal.signal(signal.SIGINT, self._signal_handler)
self.start_thread()
def set_tempo(self, tempo):
self.tempo = tempo
self.wait_mult = 60 / tempo
def _signal_handler(self, signum, frame):
"""Handle Ctrl+C signal"""
print("\nReceived Ctrl+C, stopping routines...")
self.stop_routs()
def start_thread(self):
"""Start the asyncio event loop in a separate thread"""
def run_event_loop():
self.loop = asyncio.new_event_loop()
asyncio.set_event_loop(self.loop)
self.running = True
print(f"Asyncio thread started: {threading.get_ident()}")
try:
self.loop.run_forever()
finally:
self.loop.close()
self.running = False
print(f"Asyncio thread stopped: {threading.get_ident()}")
if self.thread is not None and self.thread.is_alive():
return self.thread
self.thread = threading.Thread(target=run_event_loop, daemon=True)
self.thread.start()
# Wait for the loop to be ready
while not self.running:
time.sleep(0.01)
return self.thread
def sched(self, coro):
"""Add a coroutine to the running event loop"""
# any time a new event is scheduled, clear the routs list of finished coroutines
print(len(self.routines))
for i in range(len(self.routines)-1,-1,-1):
print(self.routines[i].done())
if self.routines[i].done():
del self.routines[i]
if not self.running or not self.loop:
raise RuntimeError("Asyncio thread is not running")
rout = asyncio.run_coroutine_threadsafe(coro, self.loop)
self.routines.append(rout)
return rout
def stop_routs(self):
"""Stop all running routines"""
for rout in self.routines:
rout.cancel()
self.routines.clear()
def get_routs(self):
"""Get all running routines"""
return self.routines
def stop_thread(self):
"""Stop the asyncio event loop and thread and start a new one"""
if self.loop and self.running:
self.loop.call_soon_threadsafe(self.loop.stop)
if self.thread:
self.thread.join(timeout=5)
self.start_thread()
# from asyncio import events
async def tc_sleep(self, delay, result=None):
"""Coroutine that completes after a given time (in seconds)."""
if delay <= 0:
await asyncio.tasks.__sleep0()
return result
delay *= self.wait_mult # Adjust delay based on tempo
loop = asyncio.events.get_running_loop()
future = loop.create_future()
h = loop.call_later(delay,
asyncio.futures._set_result_unless_cancelled,
future, result)
try:
return await future
finally:
h.cancel()
# Example using standard clock
async def counter(name, wait):
"""A counter coroutine"""
i = 0
while True:
print(f"{name}: {i}")
# uses the normal asyncio clock
await asyncio.sleep(wait)
i += 1
# Example using tempo clock
async def tc_counter(scheduler, name, wait):
"""A counter coroutine"""
i = 0
while True:
print(f"{name}: {i}")
# uses the tempo clock sleep function found in the class instead of the standard asyncio.sleep
await scheduler.tc_sleep(wait)
i += 1
# Start the asyncio thread
scheduler = Scheduler()
# scheduling on the tempo clock requires that the function has reference to the scheduler
rout1 = scheduler.sched(tc_counter(scheduler, "Counter-1", 0.5))
scheduler.set_tempo(120)
scheduler.set_tempo(25)
# scheduling on the time clock doesn't require reference to the scheduler
rout2 = scheduler.sched(counter("Counter-2", 1))
@Spacechild1, just wanted to say thanks for this great explanation. It took me a while to wrap my head around why logical time is useful. At first I thought it was only useful if you’re using the server, so that you could have notes play simultaneously even if it takes time to output them from sclang. But then I realized that it’s useful even if you’re not using the server: for example, you generate a bunch of notes in a function that are meant to be simultaneous, and you send them one by one to an object that records their onTime. If you then use the onTime to compare certain aspects of these notes, and that comparison is very fine, then even small differences in recorded onTime will have an effect. Using logical time, all notes meant to be simultaneous will have the exact same onTime recorded.
All of this makes me appreciate SC’s clocks all the more — and reinforces that it wouldn’t be trivial to create a direct equivalent in Python.
Cool!
All of this makes me appreciate SC’s clocks all the more — and reinforces that it wouldn’t be trivial to create a direct equivalent in Python.
You only need to follow the algorithm outlined in SC-like clocks in Python? - #12 by Spacechild1. I have written such schedulers in several languages, including JS. I don’t see why it should be difficult in Python. All you need is a priority queue and some function to measure the current system time.
Just a small note on the priority queue: typical implementations don’t guarantee a stable ordering of elements of equal priority (= with the same timestamp) out of the box. This means that you also need to track the insertion order. This is typically done with a counter and an additional integer member for the queue element. The comparison function can then use this member to discern elements with equal timestamps.
A very silly clock-related question, slightly OT and nothing to do with python: I’ve been messing around with periodically changing clock tempo on patterns to get accelerandi and “rubato” like phrasing. It’s a very crude approach (probably better to use stretch), but I wonder how frequently the tempo can theoretically be updated (and how frequently is it reasonable to do so)?
(—don’t tell me it depends on my use case, because my use case is nothing but pure speculation)
There is no real update limit. Just be aware that tempo changes do not take effect immediately, but only when the current thread goes to sleep. This is because every TempoClock has its own priority queue and runs in its own OS thread. The AppClock thread, SystemClock thread and the TempoClock threads never run in parallel; they all grab the global interpreter lock when they want to execute sclang code.
(and how frequently is it reasonable to do so)?
Just don’t update it faster than your perceptial limit 
Ouch, “semi-abandoned”. She’s lacking documentation, yes, but I’m working on it.
2 Likes
Definitely didn’t mean to offend. There may not even be a need for documentation per se, more just some basic examples showing how to use the various features, including clocks.
1 Like