This is way outside anything I do so this is really spitballing.
But if I was to approach this I would probably use demand side UGens that send out my patterns, that were clocked to Timer. And then use patterns to find buffers that were read by the demand side UGens. And then you could use the clock as a trigger for checking those buffers in some way. So that clock should send that signal to an audio bus that everything else checks (make sure the clock is at the beginning of that bus).
So conceptually the idea would be that patterns send updates on some regular basis for the next clock cycle (whatever that is - 1/4 note, 1/8 note - whatever) - and your synths check the new buffers on the next clock cycle.
Also, remember that you can dynamically create synths whenever (and those synths can sync themselves to the clock). So you could just create sequence patterns dynamically (maybe using the functional composition patterns), and trigger when those are created using patterns.
But my guess is that you’re going to have to play around with this.
Also you may not need to use audio rate for control rate stuff - control rate may be fast enough. That will make your life easier in some ways.