Re: Confusion between Frame-Sizes vs categorical DSP contexts

Continuing a secondary topic from this discussion by @Dasha

That is a limitation of all real-time audio/dsp environments.

If you want to avoid a frame-based approach, there is no problem in rewriting the code as a sample-based operation. The downside is that there is a price, you will use most of your CPU, and you won’t be able to vectorize and use SIMD instructions. And in the end, it won’t work. It will not be a professional system.

You have some options. You can write that some units in faust (they have an option for single-sample, but it is still “frame-based”; it’s just numFrames=1 [tongue in cheek]

… or write in a lazy functional language, for example, using Arrows in Haskell, and render it in non-real-time. You can do fun things with that; the paradigm would be different. Here, de facto is a different paradigm.

https://en.wikibooks.org/wiki/Haskell/Understanding_arrows


import Control.Arrow

{- | SF represents a signal function that takes an input of type a and produces
an output of type b along with a new signal function for subsequent inputs.
-}
newtype SF a b = SF {runSF :: (a -> (b, SF a b))}

-- |  feedback loops instance for Arrows 
instance ArrowLoop SF where
    loop sf = SF (g sf)
      where
        g f x = f' `seq` (y, SF (g f'))
          where
            ((y, z), f') = runSF f (x, z)

-- |  one-step delay with an initial value i.
instance ArrowCircuit SF where
    delay i = SF (f i)
      where
        f i x = (i, SF (f x))

This can do nice things; I’m writing stuff with this right now.

But remember: Signals, Arrows, and similar types and categories in Haskell are lazy evaluated, which allows higher-level operation with signals, BUT is (apparently) incompatible with the strictness of traditional professional audio code.

How do you build a lazy, complex, real-time system? It doesn’t even make much sense.

Implementing such an extensive system would be difficult (in practical terms). However, it is possible (the article in the footnote is just about that) to combine laziness and strictness through calculi equivalence in a more general category and supported by “translations” through new ideas that were formalized not long ago.

So, any implementation can use a call-by-need context, and the system would be capable of any necessary translation.

Still, the design must be pretty good for that to work; not only will translations be correct implemented in the right places in the right way, but the design of such a system is a blank page. This is another world in terms of qualitative implementation: a formal way to handle lazy evaluation in a functional programming context into a “normal” evaluation. etc. etc. usw

When discussing the matter properly, the frame-based process loop with frameSize=1 is irrelevant. If this is unclear, you must explore this discussion more patiently because it is something else, at the risk of mixing (some more, some less) unrelated things.

  • Reference

Maraist, J., Odersky, M., Turner, D. N., & Wadler, P. “Call-by-name, call-by-value, call-by-need, and the linear lambda calculus”


  • Side note

The frame-based approach, with process loops and audio callbacks in C and C++, is virtually how professional audio is done.

But mixing different frame sizes in the same DSP graph, with some restrictions (let’s say the number of samples must be the power of 2), is perfectly possible now, and Nodoby does. One must develop a good design and grammar to combine different numFrames.

This idea has yet to inspire people to hack an environment like that. It’s not that hard. Someone?


Some questions for basic understanding, as (I think) this is sort of around the edges of some stuff I’ve been working on, but certainly not a strength or key focus of mine.

IIUC, you’re referring to something similar to ~gen, right? Where patches on a sample level are able to be incorporated into a larger patch without changing the universal block size? I’m unsure if block size and numFrames which you’re using are the same thing, sorry for confusion.

It seems to me from the last few years on this forum that this is one of the community’s main interests/desires for the future. I sort of get around some of this stuff by hacking together C++ plugins but obviously it’d be nicer to be do everything in SC.

I would be quite interested in contributing to such a development somehow (with my very basic skills and whatever I can learn) but my understanding of ~gen is that it was a pretty monumental achievement, and such an overhaul of the SC server would be a similarly massive task?

Sorry if I’ve misunderstood and am off-topic :slight_smile:

1 Like

I don’t know anything about ~gen. Can you point to some source with the relevant information?

The link on Arrows introduces things, but if they need to be more explicit, I’m glad to supplement them later with more context information. I never know what to assume regarding the background here; we’re too diverse.

I miswrote, it’s gen~ - it’s an extension (AIUI) of Max/MSP which allows single-sample feedback loops etc. inside bigger patches. @dietcv is the local expert here :slight_smile:

I’ve never used Haskell (unless a few times messing around with Tidal counts, haha) and don’t quite get Arrows yet, but I’ll dig into the article and see if I get anywhere.

1 Like

There is also Paul Hudak’s book (Haskell School of Music). Read that now; it’s the best option for an easy-entry introduction based on music and fundamental functional programming ideas.

Arrow is a topic sort of advanced.

Note: I forked (just because there is no way to change many things without breaking compatibility, and the result might be very different, but all breaking changes are in separate branches for now) Haskore/Euterpea and am rewriting the event (soft) RT (initially used for MIDI, but of course, I will focus more on scsynth backend, using the mature haskell clients already existent, no need to do this crazy work again) and implementing a real-time scheduler for audio good enough to test small/med things. The render NRT using Doubles (WAVE_FORMAT_IEEE_DOUBLE ) will always be better for a polish run. When I have something to share regarding documentation (Arrow, Reactive Programming, etc.), I can send it to you. There is no deadline, though.

I suspect that gen~ is not about the same ideas. But let’s see, sometimes we are surprised.

@smoge I think I kind of understand what you say by lazy evaluation and context,“the call by need”, but here Im really a noob. I tried Haskell several years ago and didn’t go to far, so Monads and Arrows are complete strangers to me. I will dive into and see if I can catch up with this.

Thanks for the references :slight_smile:

1 Like

I’m working a prototype for reblocking and resampling on a per-Synth basis. I think it can be done. I’m planning to present a proof-of-concept before the SuperCollider Symposium.

6 Likes

yea NOW WE ARE TALKING !!! @Spacechild1

1 Like

I looked at the code example in the image and tried to read something in the interview.

In the interview, he defines gen~ as a “microworld,” but it is not a thing in informatics. This code looks like every professional c++ dsp code’s regular process loop, but there is only one instead of 64 or whatever samples per block. That’s precisely what I said before; it is still a frame-based technique, with block or “frame size” = 1. Maybe they have some fancy compilers, but that does not change much under the aspect I noted.

Indeed, this project has the resources to implement many more things. It’s a successful enterprise, and features must be included often to keep the business running. Maybe it just happened that this interview does not reveal anything other than that. IDK

It would be better to read something more technical.
Or could someone explain this stuff in a few words here?