Live Input TimeStretching

hey,
is there a possibilty to do timestretching on a live input signal? somehow similiar to the Ableton complex or texture warping modes? Does somebody know if the complex one is FFT based?
maybe with a BufWr / BufRd setup and this SynthDef which was once on the mailing list?
any suggestions to get me started?
thanks

(
SynthDef(\buffer_stretch_shift, { |out = 0, in = 0, midiShift = 0, stretch = 1,
	pitchDispE = -4, pitchDispM = 1, timeDispE = -2, timeDispM = 2,
	amp = 0.1|

	var sig, inSig, pitchDisp, timeDisp;
	
	inSig = PlayBuf.ar(1, b, 1 / stretch, loop: 1) ! 2;

	pitchDisp = pitchDispM * (10 ** pitchDispE);
	timeDisp = timeDispM * (10 ** timeDispE);

	sig = PitchShift.ar(
        	inSig,
        	pitchRatio: midiShift.midiratio * stretch,
        	pitchDispersion: pitchDisp,
        	timeDispersion: timeDisp
	);
	XOut.ar(out, 1, sig * amp);
}).add;
)

Yep! you can use Warp1 where you record into a buffer, and your pointer is pointing into time you have already recorded into… There is also WarpIn in the extension Quarks that handles recording and playback for you.

Be aware, though, that it’s literally impossible to run this as a long-term effect.

Let’s say r is the time ratio – r > 1 means to play back faster than normal, r < 1 is to play back slower.

If r > 1, even if you start with some delay, eventually you will need to play back audio from the future. For obvious reasons, SC doesn’t have this feature.

If r < 1, then the amount of audio from the past that needs to be remembered is continually growing – so available memory is a hard limit on the duration that the effect can run.

You can do it if you start the effect at a specific time when needed, and run it for a predetermined length of time, and then stop or reset. You can’t run it continuously for an indefinite length of time.

hjh

hey, thanks for the replies and poiting out the limitations. I will test the WarpIn Ugen (thats also granular based right?) and otherwise stick to recorded material and continue to use the ableton complex one.

It is Granular. However - if you take the time to add in the extra overhead of using Warp1 (with the time pointer) you are able to use that same pointer to drive PV_BufRd (with a PV_RecordBuf running along side RecordBuf) - I often do this so I can mix the granular and FFT based time stretching and they tend to mask each others problems well - the PV ugens mask out the granular flutter, the granular Warp1 masks out the problems with transients.

4 Likes

hey, thanks for pointing out the pros/cons of granular vs. FFT based time stretching. It seems you already have a solution which works for you. are you maybe willing to share some code?
Especially when it comes to setting up writing and reading im not sure how to do it in the best possible way.
Ive seen people using different Synthdefs and Busses or i know one Granular Reverb example based on GrainBufJ and LocalBufs in only one SynthDef with a formula for making sure reading and writing are not crossing. whats best practice here?

I’ll put together some samples tonight… ‘best practice’ in SC terms is one I avoid. But I will share the ‘josh practice’. (which I feel comfortable doing on this topic, since I wrote Warp1, the PV_Buf ugens and GrainBuf, etc - the J refers to the original GrainBuf that I wrote in the Josh UGens, before we pulled versions in as ‘official SC Grain plugins’ - and it gave an easy to update change if you want to keep old interfaces and behavior - just had to add J to your Grain UGens after the transition!)

2 Likes

that sounds great. thank you very much :slight_smile:

1 Like

(sorry! work went late last night… I’ll try to get to this today)

dont worry. im looking forward when you find some time :slight_smile:

Hi dietcv,
here is some code I had lying around exploring Joshs great Warp1 Ugen to freeze and scrub samples live with a small GUI, maybe you’ll find it useful. The live input in this case is the default sample or a file you open but if you add any other Synth with an \addToHead addAction then the \freeze SynthDef will use it as an input.

It does not know where in the buffer to start playing to get the effect that time freezes exactly in sync with the live input nor does it continue to record the input when it is “freezing”. Maybe Josh will touch upon those things in his examples.

(
s.waitForBoot{

b = Buffer.read(s, Platform.resourceDir +/+ "sounds/a11wlk01.wav");

SynthDef(\playExample, { arg out = 0, bufnum;
    Out.ar( out,
        PlayBuf.ar(1, bufnum, BufRateScale.kr(bufnum)!2, loop:1)
    )
}).add;

SynthDef(\freeze,{ arg out= 0, point= 0, pitch= 1, glide= 0.3, gate= 0, wSize= 0.1, playback= 0, rate= 0.1;
	var bufnum, signal, input, mix;
	input= In.ar(out,2);
	bufnum= LocalBuf(s.sampleRate*2, 2); // recording 2 seconds

	RecordBuf.ar(input, bufnum, 0, 1, 0, (gate-1).abs, 1, gate);
	point = [Lag.kr(point, glide), LFSaw.kr(rate).abs, SinOsc.kr(rate).abs, LFTri.kr(rate).abs, LFNoise1.kr(rate, 1, 1)/2];
	signal = Warp1.ar(2, bufnum, Select.kr(playback, point), pitch, wSize, -1, 8, 0.1, 2);

	mix= gate.linlin(0, 1, -1, 1);
	ReplaceOut.ar(out, XFade2.ar(input, signal, mix)) ;
}).add;

w= Window("Freeze").setTopLeftBounds(Rect(200, 200, 400, 200)).front;

Button(w, Rect(40, 10, 80, 20))
.states_([["open file"]])
.action_({|v|
	Dialog.openPanel({ arg path;
		path.postln;
		{
		{b.free}.try;
		s.sync;
		b = Buffer.read(s, path);
		}.fork;
	},{
  	  "cancelled".postln;
	});
});

Button(w, Rect(130, 10, 80, 20))
.states_([["play"],["stop"]])
.action_({|v|
	if(v.value == 1, {
		x= Synth(\playExample, [\bufnum, b],  addAction: \addToHead);
		a= Synth(\freeze, addAction: \addToTail);
		},{
		x.free;
		a.free;
	});
});

Button(w, Rect(220, 10, 80, 20))
.states_([["freeze"],["unfreeze"]])
.action_({|v|
	if(v.value == 1, 
		{a.set(\gate, 1)}, 
		{a.set(\gate, 0)});
});

EZSlider(w,
	Rect(0, 40, 400, 20),
	"point",
	ControlSpec(0,1, \lin, 0, 0),
	{|ez| //action
		a.set(\point, ez.value)
		},
	0, // init value
	labelWidth: 40
);

EZSlider(w,
	Rect(0, 60, 400, 20),
	"pitch",
	ControlSpec(0,2, \lin, 0, 1),
	{|ez| //action
		a.set(\pitch, ez.value)
		},
	1, // init value
	labelWidth: 40
);

EZSlider(w,
	Rect(0, 80, 400, 20),
	"glide",
	ControlSpec(0,1, \lin, 0, 0.3),
	{|ez| //action
		a.set(\glide, ez.value)
		},
	0.3, // init value
	labelWidth: 40
);

EZSlider(w,
	Rect(0, 100, 400, 20),
	"grain size",
	ControlSpec(0,1, \lin, 0, 0.3),
	{|ez| //action
		a.set(\wSize, ez.value)
		},
	0.3, // init value
	labelWidth: 40
);

EZPopUpMenu(w, Rect(20, 130, 250, 20),
	"scrub with ugen:",
	["point slider", "LFsaw", "SinOsc", "LFTri", "LFNoise2" ],
	{ |ez|
	a.set(\playback, ez.value);
	ez.value.postln;
	},
	labelWidth: 120
);

EZSlider(w,
	Rect(0, 150, 400, 20),
	"rate",
	ControlSpec(0.01, 5, \exp, 0.01, 0.1),
	{|ez| //action
		a.set(\rate, ez.value)
		},
	0.1, // init value
	labelWidth: 40
);

w.onClose= {b.free; {x.free}.try; {a.free}.try; };

}
)
3 Likes

Sorry about the delay - here is an example that creates stretched notes that chase after you (to keep up in time - so, a note start and takes the buffer just created and stretches it out, waits a few seconds, does it again). This has 2 minutes of buffer time, but you can adjust that (and the balance between FFT and Warp1) in the args… hope that helps!

(
s.options.memSize_(32768);
s.waitForBoot({
	var fftBuf1, fftBuf2, fftRecordBuf, liveInputBuf;
	var bufTime, iBufTime, fftSize, hop;
	var startTime;

	bufTime = 120.0; // in seconds
	iBufTime = bufTime.reciprocal; // handy for the time pointers
	fftSize = 4096; // must be power of 2... larger means more delay / offset
	hop = 0.25;
	liveInputBuf = Buffer.alloc(s, bufTime * s.sampleRate);
	fftRecordBuf = Buffer.alloc(s, bufTime.calcPVRecSize(fftSize, 0.25, s.sampleRate));
	s.sync;
	"Buffers loaded".postln;

	// for cleaning up
	CmdPeriod.doOnce({
		liveInputBuf.free;
		fftRecordBuf.free;
		"Buffers Freed".postln;
	});

	SynthDef(\recordInput, {arg inBus, recordBuffer, fftRecordBuffer, fftSize;
		var in, chain;
		in = Limiter.ar(In.ar(inBus, 1), -3.dbamp);
		chain = FFT(LocalBuf.new(fftSize), in, hop, 1);
		PV_RecordBuf(chain, fftRecordBuffer);
		RecordBuf.ar(in, recordBuffer);
	}).add;

	SynthDef(\stretch, {arg recordBuffer, fftRecordBuffer, fftSize, warpScale = 1.0,
		fftScale = 1.0, pan, startPoint, endPoint, duration, outBus;
		var warp, fftStretch, out, pointer, env, envGen, chain;
		pointer = Line.kr(startPoint, endPoint, duration);
		env = Env([0, 1, 1, 0], [0.1, 0.8, 0.1], \sin);
		envGen = EnvGen.kr(env, timeScale: duration, doneAction: 2);
		warp = Warp1.ar(1, recordBuffer, pointer, 1, 0.11, -1, 8, 0.1) * warpScale;
		fftStretch = 0.0;
		chain = PV_BufRd(LocalBuf.new(fftSize), fftRecordBuffer, pointer);
		fftStretch = IFFT.ar(chain) * fftScale;
		out = Pan2.ar((warp + fftStretch) * envGen, pan);
		Out.ar(outBus, out);
	}).add;


	// start recording... 1st input
	Synth(\recordInput, [\inBus, s.options.numOutputBusChannels,
		\recordBuffer, liveInputBuf,
		\fftRecordBuffer, fftRecordBuf,
		\fftSize, fftSize
	]);
	Task({
		var winSize = 12.0, overlaps = 4, stretch = 0.1; // stretch < 1 is stretched out
		var outBus = 0;
		s.sync;
		startTime = Main.elapsedTime;

		inf.do({arg i;
			var waitTime, now, noteOffset, start, end, pan;
			"New note".postln;
			now = Main.elapsedTime - startTime; // how long has this been playing?
			noteOffset = 0.1; // start our stretch a little before now to avoid glitches
			start = now - noteOffset;
			end = start + (winSize * stretch); // we want our pointer calculations done here
			start = start * iBufTime; // find where in the buffer to point to
			end = end * iBufTime; // find where in the buffer to point to
			pan = 1.0.rand2; // random pan for funsies
			Synth(\stretch, [
				\recordBuffer, liveInputBuf,
				\fftRecordBuffer, fftRecordBuf,
				\fftSize, fftSize,
				\warpScale, -6.dbamp, // futz with these numbers to find a good balance for your input
				\fftScale, -3.dbamp,
				\pan, pan,
				\startPoint, start,
				\endPoint, end,
				\duration, winSize,
				\outBus, outBus]);
			waitTime = winSize / overlaps;
			waitTime.wait;
		})
	}).play;
})
)
5 Likes

I’ll also note: this will loop and you can play for much longer than 2 minutes! But that’s the amount of space you have to fill up ‘from the past’ - more or less.

Thanks for sharing this! I was about to ask the same question as @dietcv .

Thank you very much @blindmanonacid @josh i will have a look at both examples :slight_smile:

both examples work great :slight_smile: im still trying to set this up with PbindFx instead of using a Task, to be able to sequence the freezing per event. kind of tricky.