STFT on language side

Questions
1

Is it possible to read in an audio file, run the STFT on the language side (rather than using UGens), and copy the STFT data to a server-side buffer? Or do I have to use the FFT ugen and PV_RecordBuf?, and wait for it to read through the audio file?

2

The Signal class has .fft and .ifft. See Signal | SuperCollider 3.12.2 Help

Hope that helps,
Paul

3

To transmit to the server, you need to re-order the data into FFT / IFFT’s format.

f = SoundFile.openRead(Platform.resourceDir +/+ "sounds/a11wlk01.wav");
f.seek(140000);
d = Signal.newClear(32768);
f.readData(d);
f.close;

f = d.fft(Signal.newClear(d.size), Signal.fftCosTable(d.size));

(
~pvPack = { |fft|
    var x = fft.real, y = fft.imag;
    var halfSize = x.size div: 2;
    var out = Signal.newClear(x.size);
    out[0] = x[0];  // DC real value (imag always = 0)
    out[1] = x[halfSize];  // Nyquist real value (imag always = 0)
    (1..halfSize-1).do { |i|
        out[i * 2] = x[i];  // then interleave real, imag pairs
        out[i * 2 + 1] = y[i];
    };
    out
};
)

~laced = ~pvPack.(f);

s.boot;

b = Buffer.sendCollection(s, ~laced, 1);

(
a = {
    // polar: 0 because I sent complex data
    // change this to 1 if you sent mag, phase pairs instead of real, imag
    var kernel = FFTTrigger(b, polar: 0);
    // to render directly, you must copy
    // because IFFT overwrites its buffer
    // (it assumes an earlier FFT had populated it)
    var fft = PV_Copy(kernel, LocalBuf(b.numFrames));
    (IFFT(fft) * 0.1).dup
}.play;
)

(
a = {
    var kernel = FFTTrigger(b, polar: 0);
    var sig = SinOsc.ar(LFDNoise0.kr(25).exprange(50, 8000));
    var fft = FFT(LocalBuf(b.numFrames).clear, sig);
    // no need to copy because 'fft' is the volatile buffer
    fft = PV_Mul(fft, kernel);  // (convolution)
    // yeah, the resynthesis really is that loud
    (IFFT(fft) * 0.0015).dup
}.play;
)
    
a.free;

hjh

1 Like
4

thanks! This is running the FFT on the entire signal, right? Is there a way to run the STFT that doesn’t involve explicitly coding the hopping part and windowing?

5

No. Signal:fft assumes that you are giving it an already-prepared single window of audio.

There might be a quark somewhere, but I don’t know.

Also, PV_RecordBuf’s format is different from FFT’s, but I forget exactly what it is (and don’t have time at the moment to check the sc3-plugins source code).

hjh

6

A solution is in FluCoMa: FluidBufSTFT takes a mono buffer of audio and gives you the stft in another buffer.

2 Likes