Is it possible to read in an audio file, run the STFT on the language side (rather than using UGens), and copy the STFT data to a server-side buffer? Or do I have to use the FFT ugen and PV_RecordBuf?, and wait for it to read through the audio file?
To transmit to the server, you need to re-order the data into FFT / IFFT’s format.
f = SoundFile.openRead(Platform.resourceDir +/+ "sounds/a11wlk01.wav");
f.seek(140000);
d = Signal.newClear(32768);
f.readData(d);
f.close;
f = d.fft(Signal.newClear(d.size), Signal.fftCosTable(d.size));
(
~pvPack = { |fft|
var x = fft.real, y = fft.imag;
var halfSize = x.size div: 2;
var out = Signal.newClear(x.size);
out[0] = x[0]; // DC real value (imag always = 0)
out[1] = x[halfSize]; // Nyquist real value (imag always = 0)
(1..halfSize-1).do { |i|
out[i * 2] = x[i]; // then interleave real, imag pairs
out[i * 2 + 1] = y[i];
};
out
};
)
~laced = ~pvPack.(f);
s.boot;
b = Buffer.sendCollection(s, ~laced, 1);
(
a = {
// polar: 0 because I sent complex data
// change this to 1 if you sent mag, phase pairs instead of real, imag
var kernel = FFTTrigger(b, polar: 0);
// to render directly, you must copy
// because IFFT overwrites its buffer
// (it assumes an earlier FFT had populated it)
var fft = PV_Copy(kernel, LocalBuf(b.numFrames));
(IFFT(fft) * 0.1).dup
}.play;
)
(
a = {
var kernel = FFTTrigger(b, polar: 0);
var sig = SinOsc.ar(LFDNoise0.kr(25).exprange(50, 8000));
var fft = FFT(LocalBuf(b.numFrames).clear, sig);
// no need to copy because 'fft' is the volatile buffer
fft = PV_Mul(fft, kernel); // (convolution)
// yeah, the resynthesis really is that loud
(IFFT(fft) * 0.0015).dup
}.play;
)
a.free;
hjh
1 Like
thanks! This is running the FFT on the entire signal, right? Is there a way to run the STFT that doesn’t involve explicitly coding the hopping part and windowing?
No. Signal:fft assumes that you are giving it an already-prepared single window of audio.
There might be a quark somewhere, but I don’t know.
Also, PV_RecordBuf’s format is different from FFT’s, but I forget exactly what it is (and don’t have time at the moment to check the sc3-plugins source code).
hjh
A solution is in FluCoMa: FluidBufSTFT takes a mono buffer of audio and gives you the stft in another buffer.
2 Likes