iFFT Freeze/Scrub

Quick exploratory (very much noob) question: is it possible to “freeze” and arbitrarily “scrub” through a buffer of FFT data?

here’s a SynthDef copied from somewhere in the docs IIRC

		SynthDef("pvrec", { |recBuf=1 soundBufnum=2 fftSize=2048 hop= 0.5 window=1|
			var in, chain, bufnum;
			bufnum = LocalBuf.new(fftSize);
			Line.kr(1, 1, BufDur.kr(soundBufnum), doneAction: 2);
			in = PlayBuf.ar(1, soundBufnum, 1, loop: 0);
			chain = FFT(bufnum, in, hop, window);
			chain = PV_RecordBuf(chain, recBuf, 0, 1, 0, hop, window);
			// no ouput ... simply save the analysis to recBuf

…and heres one that plays it back - you can fiddle with the rate (negative ok) -

	SynthDef("pvplay", { | out=0, recBuf=1, rate=1 fftSize=2048 hop= 0.5 window = 1|
			var in, chain, bufnum;
			bufnum = LocalBuf.new(fftSize);
			chain = PV_PlayBuf(bufnum, recBuf, rate, 0, 1, 1, 0.25, 1);
			Out.ar(out, IFFT(chain, window).dup);

1 Like

Great, thank you very much @semiquaver! I will give that a go.

Another quick question: is it possible to transpose the resynthesised FFT data to a pitch scale (ie up/down in semitones)?

None of the FFT uGens seem designed for this particular task to my untrained eye, based on their descriptions, but I may be missing something.

Ah, this one, I guess:

But is it necessary to stretch/compress the bin locations, as well as offsetting them up/down?

I will experiment. Need to do some reading :wink:

Is it possible to analyse an audio file to an FFT data buffer in faster than realtime (your example plays back audio and analyses it in realtime to the FFT buffer, as I understand it)?

One possibility: check out @scztt Offline Process Quark

(though now that I look I see a complication that the quark expects to produce audio rate file - not sure if side effects are produced! )

The output file is not necessarily audio rate - it writes to an audio file, but that’s only because the only output format NRT supports is audio files. If you e.g. specify a control rate process, you’ll get an audio file w/ control rate data - likewise with outputting triggers via putTr. The quark will read these audio files back into sclang as arrays, so you don’t need to interact with the on-disk audio files at all unless you explicitly need to.

1 Like

it was the language: “Offlice Process expects audio rate output Ugen” in the help that made me wonder - the OP is hoping to store a PV chain - possible?

(and something like this should really be included in SC imo)

Here’s an example of capturing an FFT chain:

~o = OfflineProcess();
~o.putTrig(\fft, {
	var fft, magsPhases;
	fft = FFT(LocalBuf(256), in);
	magsPhases = UnpackFFT(fft, 256);
	[fft, magsPhases];

fork {
	~run = ~o.process(Platform.resourceDir +/+ "sounds/a11wlk01.wav");

fft in your SynthDef is actually a trigger that is fired every time there’s a new FFT buffer available, so you can use it as the trigger for putTrig. One caveat here is that the above won’t naively work for FFT sizes larger than 256 - I think this might be a limitation of UnpackFFT? You may need to use multiple UnpackFFT's, or roll your own with an array of Unpack1FFT.

1 Like

I get

“Execution warning: Class ‘Deferred’ not found
ERROR: Message ‘new’ not understood.”

After compiling your Quark and trying that code

Deferred quark also on szctt github IIRC

The OfflineProcess looks interesting, but it wouldn’t result in a buffer that could be used with PV_PlayBuf. So, playing back may be difficult.

You can run a PV_RecordBuf synth in a NRT server.

p = Platform.resourceDir +/+ "sounds/a11wlk01.wav";
q = "~/pvtest.wav".standardizePath;

// get duration
f = SoundFile.openRead(p);
f.duration  // 4.2832879818594

var resultbuf, inbuf;
var fftSize = 512;

z = Server(\nrt, NetAddr("", 57110),

inbuf = Buffer(z, 65536, 1);
resultbuf = Buffer(z, f.duration.calcPVRecSize(fftSize, 0.5, z.options.sampleRate), 1);

x = Score([
	[0, inbuf.allocMsg],
	[0, resultbuf.allocMsg],
	[0, inbuf.readMsg(p, leaveOpen: true)],
	[0, [\d_recv, SynthDef(\pv_ana, {
		var sig = VDiskIn.ar(1, inbuf, f.sampleRate / SampleRate.ir);
		var fft = FFT(LocalBuf(fftSize, 1), sig);
		fft = PV_RecordBuf(fft, resultbuf, run: 1);
		Out.ar(0, sig);
	[0, Synth.basicNew(\pv_ana, z).newMsg],
	[f.duration + (fftSize / z.options.sampleRate),
		resultbuf.writeMsg(q, "wav", "float")

	outputFilePath: if(thisProcess.platform.name == \windows) { "NUL" } { "/dev/null" },
	headerFormat: "wav", sampleRate: z.options.sampleRate,
	options: z.options,
	duration: f.duration + (fftSize / z.options.sampleRate),
	action: { "done".postln }


And play it back:


// this is the pv_rec result
b = Buffer.read(s, q);

SynthDef(\pvmouse, { |out = 0, recBuf = 1, fftSize = 512|
	var in, chain, bufnum;
	bufnum = LocalBuf.new(fftSize);
	chain = PV_BufRd(bufnum, recBuf, MouseX.kr(0, 1));
	Out.ar(out, IFFT(chain).dup);

a = Synth(\pvmouse, [recBuf: b]);



This got over my head quite quickly. Have to read up on some basics, I think. Good to now there are solutions, though. Thanks very much guys!

My NRT example is mostly based on an example in the help: Non-Realtime Synthesis (NRT) | SuperCollider 3.11.1 Help

NRT is very different from normal real-time usage. It does make sense but it takes some time to understand the differences. The linked help file tries to walk you through them – if you expect to want to do more of this type of processing, it’s worth it to work your way through it.


but … I ran James’ code and …so fun to scrub so thanks to you both!

1 Like

Hi @semiquaver do you have the exact code you used to hand?

Don’t worry, got it working. And it’s a LOT of fun!!

Thank you very much, guys!!!

Hi @jamshark70 I’m really enjoying developing my PV scrubbing script. Thanks so much for your example!

I wonder if it would be possible to analyse the pitch of the original audio, and save that to a separate file, in the same way as the FFT analysis, and to load that back into a buffer and scrub through it at the same time as the FFT data.

With a known fundamental frequency, I could do frequency-domain pitch-shifting/flattening, which would be really cool!

Do you think that’s possible?

It’s certainly possible.

The analysis example creates one resultBuf – there’s nothing stopping you from creating a second resultBuf for a control-rate pitch detector result. What is the size of this buffer? Control rate is one value per blockSize audio samples, so:

inbuf = Buffer(z, 65536, 1);
resultbuf = Buffer(z, f.duration.calcPVRecSize(fftSize, 0.5, z.options.sampleRate), 1);
pitchbuf = Buffer(z, (f.numFrames / z.options.blockSize).roundUp.asInteger, 2);

(2 channels because pitch detectors usually output a confidence value along with the frequency.)

Then add a second analysis and second buffer recorder into the analysis SynthDef.

SynthDef(\pv_ana, {
	var sig = VDiskIn.ar(1, inbuf, f.sampleRate / SampleRate.ir);
	var fft = FFT(LocalBuf(fftSize, 1), sig);
	var pitch = Tartini.kr(sig);
	RecordBuf.kr(pitch, pitchbuf, loop: 0);
	fft = PV_RecordBuf(fft, resultbuf, run: 1);
	Out.ar(0, sig);

And where it writes the FFT analysis buffer, add a second command to write the pitch buffer.

	[f.duration + (fftSize / z.options.sampleRate),
		resultbuf.writeMsg(q, "wav", "float")
	[f.duration + (fftSize / z.options.sampleRate),
		pitchbuf.writeMsg(q.dirname +/+ "pitchbuf.wav", "wav", "float")

One thing I would note here is that – there isn’t much new in the above advice. The example already shows how to create, fill and write an analysis buffer – and the answer to the question is just to do that with two buffers.

Or, as I’ve been preaching lately: solve one problem at a time. If you want to integrate pitch analysis into a NRT analysis code snippet, then… the integration can’t happen without having something to integrate (that is, without working out the pitch analysis by itself first – i.e., generating the pitch analysis signals, and then storing them in a buffer).

Quite often, I see people trying to handle all of the requirements in one go. That’s a good recipe to get confused.