Instantaneously IFFT into a buffer?

[Eric_Sluyter]

So I’m wondering if it’s possible, instead of playing the result of an IFFT in real time:

~fftbuf = Buffer.alloc(s, 1024);

(
{
  var in = SoundIn.ar(0);
  var chain = FFT(~fftbuf, in, 1).pvcalc(1024, { |mags, phases|
    [mags, 0!1024]
  });
  var sig = IFFT(chain);
  sig
}.play;
)

to instead instantly write (in this case) the 1024 signal points into a Buffer.

I can basically get what I want using RecordBuf:

~fftbuf = Buffer.alloc(s, 1024);
~resbuf = Buffer.alloc(s, 1024);

(
{
  var in = SoundIn.ar(0);
  var chain = FFT(~fftbuf, in, 1).pvcalc(1024, { |mags, phases|
    [mags, 0!1024]
  });
  var sig = IFFT(chain);
  
  RecordBuf.ar(sig, ~resbuf, trigger: chain);
  nil;
}.play;
)

But this introduces another 1024 samples delay before I can read the whole IFFT from the buffer, and also is constantly writing, so most of the time the buffer contains a discontinuity somewhere.

It would be better if there were something like IFFT that could write the transform immediately into a buffer… does this exist already?

Relatedly, is there a UGen that will copy the contents of one buffer into another buffer on receiving a trigger?

[Sam_Pluta]

Looking at the source, the internal sound playback buffer doesn’t get copied to the FFT buffer after the IFFT process. It is a separate internal buffer. What is your goal?

BufFFT library does things similar to what you want, but not exactly. BufFFT_BufCopy will copy on a trigger. But not like your example. It will copy into the chain when the chain triggers.

Sam

[Eric_Sluyter]

@Sam_Pluta

What is your goal?

I was hoping to, every 1024 samples, FFT the input, do some PV manipulations on it, and then IFFT into a buffer, which would be read as a wavetable by a separate phasor. (And eventually alternate between two buffers for smooth xfade)

[Eric_Sluyter]

It’s not pretty, but using BufFFT library I’ve managed to get this far…

~fftbuf = Buffer.alloc(s, 1024);
~tempbuf = Buffer.alloc(s, 1024);
~resbuf = Buffer.alloc(s, 1024);
~oldresbuf = Buffer.alloc(s, 1024);

~demobuf = Buffer.read(s, Platform.resourceDir +/+ "sounds/a11wlk01.wav");

(
{
  //var in = SoundIn.ar(0);
  var in = PlayBuf.ar(1, ~demobuf, BufRateScale.kr(~demobuf), loop: 1);
  var chain = FFT(~fftbuf, in, 1).pvcalc(1024, { |mags, phases|
    [mags, 0!1024]
  });
  var sig = IFFT(chain);
  
  var oldResChain = BufFFTTrigger(~oldresbuf);
  var resChain = BufFFTTrigger(~resbuf);
  
  var fftPhase = Phasor.ar(resChain, 1024.reciprocal);
  var readPhase = Phasor.ar(DC.ar(0), MouseX.kr(10, 1000, \exponential) * SampleDur.ir);
  
  RecordBuf.ar(sig, ~tempbuf, trigger: chain);
  oldResChain = BufFFT_BufCopy(oldResChain, ~resbuf, 0);
  resChain = BufFFT_BufCopy(resChain, ~tempbuf, 0);
  
  
  XFade2.ar(
    BufRd.ar(1, ~oldresbuf, readPhase * 1024), 
    BufRd.ar(1, ~resbuf, readPhase * 1024),
    fftPhase.linlin(0, 1, -1, 1)
  ) !2;
}.play;
)

It is glitching, which I think it shouldn’t since I’m setting all the FFT phases to 0 and both ~resbuf and ~oldresbuf look continuous / play back without glitches if I stop the real time FFT stuff… so there must be some timing problem somewhere.

(I see that BufFFTTrigger is delayed by one control sample relative to FFT and FFTTrigger, but weirdly this seems to actually improve the glitching…)

[Sam_Pluta]

This is awesome. I think it just needs an env over the BufRd. I removed the one In the IFFT and put it over the BufRd, which should be equivalent.

Unless I am misunderstanding, you need to manually add a second layer to get the overlapping windows and thus a smooth output.

(
{
  //var in = SoundIn.ar(0);
  var in = PlayBuf.ar(1, ~demobuf, BufRateScale.kr(~demobuf), loop: 1);
  var chain = FFT(~fftbuf, in, 1).pvcalc(1024, { |mags, phases|
    [mags, 0!1024]
  });
  var sig = IFFT(chain, -1);
  
  var oldResChain = BufFFTTrigger(~oldresbuf);
  var resChain = BufFFTTrigger(~resbuf);
  
  var fftPhase = Phasor.ar(resChain, 1024.reciprocal);
  var readPhase = Phasor.ar(DC.ar(0), MouseX.kr(10, 1000, \exponential) * SampleDur.ir);
	
	var env = (readPhase*pi).sin**2;
  
  RecordBuf.ar(sig, ~tempbuf, trigger: chain);
  oldResChain = BufFFT_BufCopy(oldResChain, ~resbuf, 0);
  resChain = BufFFT_BufCopy(resChain, ~tempbuf, 0);
  
	
  
  XFade2.ar(
    BufRd.ar(1, ~oldresbuf, readPhase * 1024)*env, 
    BufRd.ar(1, ~resbuf, readPhase * 1024)*env,
    fftPhase.linlin(0, 1, -1, 1)
  ) !2;
}.play;
)

[Sam_Pluta]

Also, chain is one sample ahead, trigger-wise:

(
{
  //var in = SoundIn.ar(0);
  var in = PlayBuf.ar(1, ~demobuf, BufRateScale.kr(~demobuf), loop: 1);
  var chain = FFT(~fftbuf, in, 1).pvcalc(1024, { |mags, phases|
    [mags, 0!1024]
  });
  var sig = IFFT(chain, -1);
  
  var oldResChain = BufFFTTrigger(~oldresbuf);
  var resChain = BufFFTTrigger(~resbuf);
  
	[chain, oldResChain, resChain]
}.plot(0.1);
)

[Eric_Sluyter]

@Sam_Pluta

I removed the one In the IFFT and put it over the BufRd, which should be equivalent.

Hm yeah this is why it doesn’t totally make sense to me that this version sounds somehow better… but it does, so thanks!

@Sam_Pluta

Also, chain is one sample ahead, trigger-wise

Yes I had noticed that, and yet for some reason using FFTTrigger (which lines up perfectly) instead of BufFFTTrigger makes the glitching worse… it would make more sense to me that they should all happen on the same trigger

[Eric_Sluyter]

@Sam_Pluta

Unless I am misunderstanding, you need to manually add a second layer to get the overlapping windows and thus a smooth output.

Here’s with a second layer…

(
{
  //var in = SoundIn.ar(0);
  var in = PlayBuf.ar(1, ~demobuf, BufRateScale.kr(~demobuf), loop: 1);
  var chain = FFT(~fftbuf, in, 1).pvcalc(1024, { |mags, phases|
    [mags, 0!1024]
  });
  var sig = IFFT(chain, -1);
  
  var oldResChain = BufFFTTrigger(~oldresbuf);
  var resChain = BufFFTTrigger(~resbuf);
  
  var fftPhase = Phasor.ar(resChain, 1024.reciprocal);
  var readPhase = Phasor.ar(DC.ar(0), MouseX.kr(10, 1000, \exponential) * SampleDur.ir);
  var readPhase2 = (readPhase + 0.5).wrap;
	
  var env = sin(readPhase * pi) ** 2;
  var env2 = sin(readPhase2 * pi) ** 2;
  
  RecordBuf.ar(sig, ~tempbuf, trigger: chain);
  oldResChain = BufFFT_BufCopy(oldResChain, ~resbuf, 0);
  resChain = BufFFT_BufCopy(resChain, ~tempbuf, 0);
  
	
  
  XFade2.ar(
    BufRd.ar(1, ~oldresbuf, readPhase * 1024) * env, 
    BufRd.ar(1, ~resbuf, readPhase * 1024) * env,
    fftPhase.linlin(0, 1, -1, 1)
  ) + XFade2.ar(
    BufRd.ar(1, ~oldresbuf, readPhase2 * 1024) * env2, 
    BufRd.ar(1, ~resbuf, readPhase2 * 1024) * env2,
    fftPhase.linlin(0, 1, -1, 1)
  ) 
  !2;
}.play;
)

Definitely an improvement, but still getting some small pops… which I can only think are due to the fftPhase not correctly crossfading between old and new FFT frame? but either moving the trigger for fftPhase a sample earlier or later just makes the popping worse. Or, that there might be a discontinuity in the FFT frame because of the delayed BufFFTTrigger, but as I say changing this to FFTTrigger also makes popping worse.

[Eric_Sluyter]

Actually, scoping signal against fftPhase and readPhase, I see discontinuities that aren’t on any phasor jump… hmmmm

(didn’t include readPhase2 here but I just checked and they aren’t on those jumps either)

actually every discontinuity seems to occur at the same point between 4 and 5 control samples after chain, this is plotted against Delay2.kr(Delay2.kr(chain.clip(0, 1))):

[Sam_Pluta]

BufFFTTrigger triggers one blocksize after FFT. This is because FFT triggers in the last block of the FFT size (so if the fft is 512 and the block is 64, it will trigger at sample 448).

The BufFFT stuff triggers right on the top of each FFT block, so at 0 and 512, etc.

I think replacing this in your code fixes the issue:

    //   var oldResChain = BufFFTTrigger(~oldresbuf);
    //   var resChain = BufFFTTrigger(~resbuf);
    var oldResChain = Select.kr(chain>(-1), [-1, ~oldresbuf]);
    var resChain = Select.kr(chain>(-1), [-1, ~resbuf]);

since it replaces the FFT trigger with a new buf number.

Sam

[Eric_Sluyter]

Well, I happened to restart my computer today and now I don’t hear any of the same glitches. (I had restarted SC in the meanwhile though, so it was a bigger problem. But weirdly I did other things with SC and the audio was fine, and all other audio my computer played normally…hmmm. They’re even on recordings I made of this specific thing and not other things, so I know I didn’t totally make this up)

With everything exactly as it was I do still see those little discontinuities when I plot so I guess they weren’t actually what I was hearing…

Anyway, wow I find your Select idea very clever, it still hasn’t totally eliminated the discontinuities but I have a harder time finding them, and I don’t think I hear them…

so yay and thanks for all your help. Without glitches, this is a pretty neat effect! (I heard about this idea from a lecture by Miller Puckette – he puts two copies of the IFFT side by side so you can swipe through them, which is what I’m up to now. Very fun.)

Although noting that it could still be twice as fast for real time if there were some in-place IFFT option… I am game to try to make this but honestly don’t know where to start, FFT buffer stuff feels like magic to me still. When I have more free time…