iFFT Freeze/Scrub

Quick exploratory (very much noob) question: is it possible to “freeze” and arbitrarily “scrub” through a buffer of FFT data?

here’s a SynthDef copied from somewhere in the docs IIRC

		SynthDef("pvrec", { |recBuf=1 soundBufnum=2 fftSize=2048 hop= 0.5 window=1|
			var in, chain, bufnum;
			bufnum = LocalBuf.new(fftSize);
			Line.kr(1, 1, BufDur.kr(soundBufnum), doneAction: 2);
			in = PlayBuf.ar(1, soundBufnum, 1, loop: 0);
			chain = FFT(bufnum, in, hop, window);
			chain = PV_RecordBuf(chain, recBuf, 0, 1, 0, hop, window);
			// no ouput ... simply save the analysis to recBuf
		}).add;

…and heres one that plays it back - you can fiddle with the rate (negative ok) -

	SynthDef("pvplay", { | out=0, recBuf=1, rate=1 fftSize=2048 hop= 0.5 window = 1|
			var in, chain, bufnum;
			bufnum = LocalBuf.new(fftSize);
			chain = PV_PlayBuf(bufnum, recBuf, rate, 0, 1, 1, 0.25, 1);
			Out.ar(out, IFFT(chain, window).dup);
		}).add;

1 Like

Great, thank you very much @semiquaver! I will give that a go.

Another quick question: is it possible to transpose the resynthesised FFT data to a pitch scale (ie up/down in semitones)?

None of the FFT uGens seem designed for this particular task to my untrained eye, based on their descriptions, but I may be missing something.

Ah, this one, I guess:
https://doc.sccode.org/Classes/PV_BinShift.html

But is it necessary to stretch/compress the bin locations, as well as offsetting them up/down?

I will experiment. Need to do some reading :wink:

Is it possible to analyse an audio file to an FFT data buffer in faster than realtime (your example plays back audio and analyses it in realtime to the FFT buffer, as I understand it)?

One possibility: check out @scztt Offline Process Quark

(though now that I look I see a complication that the quark expects to produce audio rate file - not sure if side effects are produced! )

The output file is not necessarily audio rate - it writes to an audio file, but that’s only because the only output format NRT supports is audio files. If you e.g. specify a control rate process, you’ll get an audio file w/ control rate data - likewise with outputting triggers via putTr. The quark will read these audio files back into sclang as arrays, so you don’t need to interact with the on-disk audio files at all unless you explicitly need to.

1 Like

it was the language: “Offlice Process expects audio rate output Ugen” in the help that made me wonder - the OP is hoping to store a PV chain - possible?

(and something like this should really be included in SC imo)

Here’s an example of capturing an FFT chain:

(
~o = OfflineProcess();
~o.putTrig(\fft, {
	|in|
	var fft, magsPhases;
	fft = FFT(LocalBuf(256), in);
	magsPhases = UnpackFFT(fft, 256);
	[fft, magsPhases];
});

fork {
	~run = ~o.process(Platform.resourceDir +/+ "sounds/a11wlk01.wav");
	~run.wait();
	
	"DONE".postln;
	~run.resultData(\fft).do(_.postln);
};
)

fft in your SynthDef is actually a trigger that is fired every time there’s a new FFT buffer available, so you can use it as the trigger for putTrig. One caveat here is that the above won’t naively work for FFT sizes larger than 256 - I think this might be a limitation of UnpackFFT? You may need to use multiple UnpackFFT's, or roll your own with an array of Unpack1FFT.

1 Like

I get

“Execution warning: Class ‘Deferred’ not found
ERROR: Message ‘new’ not understood.”

After compiling your Quark and trying that code

Deferred quark also on szctt github IIRC

The OfflineProcess looks interesting, but it wouldn’t result in a buffer that could be used with PV_PlayBuf. So, playing back may be difficult.

You can run a PV_RecordBuf synth in a NRT server.

p = Platform.resourceDir +/+ "sounds/a11wlk01.wav";
q = "~/pvtest.wav".standardizePath;

// get duration
f = SoundFile.openRead(p);
f.close;
f.duration  // 4.2832879818594

(
var resultbuf, inbuf;
var fftSize = 512;

z = Server(\nrt, NetAddr("127.0.0.1", 57110),
	ServerOptions.new
	.numOutputBusChannels_(2)
	.numInputBusChannels_(2)
	.sampleRate_(44100)
);

inbuf = Buffer(z, 65536, 1);
resultbuf = Buffer(z, f.duration.calcPVRecSize(fftSize, 0.5, z.options.sampleRate), 1);

x = Score([
	[0, inbuf.allocMsg],
	[0, resultbuf.allocMsg],
	[0, inbuf.readMsg(p, leaveOpen: true)],
	[0, [\d_recv, SynthDef(\pv_ana, {
		var sig = VDiskIn.ar(1, inbuf, f.sampleRate / SampleRate.ir);
		var fft = FFT(LocalBuf(fftSize, 1), sig);
		fft = PV_RecordBuf(fft, resultbuf, run: 1);
		Out.ar(0, sig);
	}).asBytes]],
	[0, Synth.basicNew(\pv_ana, z).newMsg],
	[f.duration + (fftSize / z.options.sampleRate),
		resultbuf.writeMsg(q, "wav", "float")
	]
]);

x.recordNRT(
	outputFilePath: if(thisProcess.platform.name == \windows) { "NUL" } { "/dev/null" },
	headerFormat: "wav", sampleRate: z.options.sampleRate,
	options: z.options,
	duration: f.duration + (fftSize / z.options.sampleRate),
	action: { "done".postln }
);

z.remove;
)

And play it back:

s.boot;

// this is the pv_rec result
b = Buffer.read(s, q);

(
SynthDef(\pvmouse, { |out = 0, recBuf = 1, fftSize = 512|
	var in, chain, bufnum;
	bufnum = LocalBuf.new(fftSize);
	chain = PV_BufRd(bufnum, recBuf, MouseX.kr(0, 1));
	Out.ar(out, IFFT(chain).dup);
}).add;
)

a = Synth(\pvmouse, [recBuf: b]);
a.free;

hjh

2 Likes

This got over my head quite quickly. Have to read up on some basics, I think. Good to now there are solutions, though. Thanks very much guys!

My NRT example is mostly based on an example in the help: Non-Realtime Synthesis (NRT) | SuperCollider 3.11.1 Help

NRT is very different from normal real-time usage. It does make sense but it takes some time to understand the differences. The linked help file tries to walk you through them – if you expect to want to do more of this type of processing, it’s worth it to work your way through it.

hjh

but … I ran James’ code and …so fun to scrub so thanks to you both!

1 Like

Hi @semiquaver do you have the exact code you used to hand?

Don’t worry, got it working. And it’s a LOT of fun!!

Thank you very much, guys!!!