Hi, is it possible to pan the elements of an FFT as individual signals in a stereo field or other multichannel setups? Meaning that every partial has its own unique pan position or movement.
I’m not aware of a single UGen to do this. You could probably hack it up using pvcollect (distribute the magnitude across multiple output FFT frames) but it would be a huge SynthDef. (Might be useful for prototyping.) That is, for each partial, you’d get magnitude and phase. Phase, you can leave alone (pass directly through to the target buffers) but the magnitude is the total energy for that partial – so you could work out n channels’ magnitude values that sum to the incoming magnitude.
The other way would be to look at the PV UGens in the source code (server/plugins directory) and write a new UGen that distributes energy from one FFT buffer into multiple FFT buffers. Then you could separately IFFT them.
It might not sound like you expect, though. IFFT resynthesizes the sounding partials from the interaction of multiple fixed-frequency sinusoidal components. Splitting them up is likely to sound cool but it probably wouldn’t be transparent panning.
hjh
Two options as far as I can see.
1
No, but you could have two pan buffers, one for left, one for right…
// buffers, all fft size;
var fft_left, fft_right, pan_left, pan_right;
PV_Copy(fft_left, fft_right);
PV_MagMul(fft_left, pan_left);
PV_MagMul(fft_right, pan_right);
Now you just have to make sure than the contents of pan_right are 1 - pan_left (or whatever pan law you want), which is probably best done in the language. I am not sure how the real and imaginary parts of the fft are place in the buffer so there is an additional issue here.
This approach is a little involved, but does offer the most control (at least in theory).
2
If you are looking to spread them around the space,but don’t case around their exact positions you might be able to use the diffuse and spread encoders in the ambisonic toolkit — FoaEncoderKernel . While obviously an fft operation, under the hood, the spreads the partial across an ambisonic field — from there it would be possible to mutate the field, moving the partials around. This is how I would approach this issue (because it is easy and will probably sound the best), but it won’t give you the finest control possible.
If the sum of the partials separated in different audio channels is identical to the original, the sound result is very interesting, closer to a scattering/refraction of sound than “normal” spatialization. It would be good to have a UGen optimized for this.
Hi,
spatialization with FFT (or band-splitting with filters) is – in my opinion – a very interesting topic. I’ve used techniques like these in several of my multichannel pieces that developed from stereo sources. I did not distribute partials of a given signal – the concept would only make sense for harmonic signals – but spectral regions. However, this gives a lots of options.
In my last “Unorthodox Synthesis” course I included a chapter with unorthodox spatialization. I’m posting one of those files below. PV_RectComb is a good starting point, the stereo examples with this PV ugen can be extended to multichannel with not much effort.
/////////////////////////////////////////
SPATIAL PROJECTION 3 – FURTHER STRATEGIES
/////////////////////////////////////////
//////////////////////////////////
Spatialization with band-splitting
//////////////////////////////////
// The following examples suppose an 8 channel ring and
// a circular clockwise ordering of channels, starting with 0 at front.
// A different starting point doesn't change the examples principally.
// Examples can be played provisionally with a stereo setup and watched with scope.
//// preparation
(
// if working with an interface you might want to use an out offset variable
// if there are different out streams (e.g. ADAT, phones, analog out)
~outOffset = 0;
s.options.numOutputBusChannels = 8;
// uncomment and adapt this for your setup, e.g.
// s.options.outDevice = "...";
// s.options.numOutputBusChannels = 30;
// s.options.outputStreamsEnabled = "11000";
// ~outOffset = 0;
s.reboot;
)
s.freqscope;
s.scope;
////////////////////////////////////////////////////////////////////
// Band-splitting with FFT is producing artefacts,
// it does not act like a perfect filter ("FFT leakage").
// This might or might not be an issue,
// alternatively you could use steep filters which are implemented by
// the classes BandSplitter2, BandSplitter4 and BandSplitter8 in the BandSpliiter quark
////////////////////////////////////////////////////////////////////
// FFT in SuperCollider
// SC contains the suite of PV (phase vocoder) UGens
// They can perform FFT analysis, manipulation and resynthesis in quasi realtime
// (latency: window size - block size)
// The FFT, IFFT and PV_ objects act differently from standard UGens
// See the help file "FFT Overview"
////////////////////////////////////////////////////////////////////
// Ex.0a
// simple spat example with PV_RectComb
// PV_RectComb produces gaps in the spectrum that are called teeth
// 'numTeeth': number of teeth
// 'phase': for shifting the teeth (not to be confused with bin phases !)
// 'width': width of teeth
// control of teeth with MouseX
(
x = {
var in, out, chain_L, chain_R, chain, numTeeth, hop = 1/2;
in = PinkNoise.ar(0.5);
// in = Saw.ar(100);
chain_L = FFT(LocalBuf(2048), in, hop: hop);
// for different processing of one buffer we must copy
// as PV UGens are writing to their buffers
chain_R = PV_Copy(chain_L, LocalBuf(2048));
numTeeth = MouseX.kr(1, 20);
// also check with phase offset
chain_L = PV_RectComb(chain_L, numTeeth, phase: 0);
chain_R = PV_RectComb(chain_R, numTeeth, phase: 0.5);
IFFT([chain_L, chain_R]) * hop;
// or more elegant:
// chain = PV_RectComb([chain_L, chain_R], numTeeth, phase: [0, 0.5]);
// IFFT(chain) * hop;
}.play(outbus: ~outOffset);
)
// observe L and R in freqcsope with linear scaling (toggle with 'BusIn' and 'FrqScl')
s.freqscope
x.release
// same with additional phase offset control by MouseY
// this means shfting the teeth while keeping them complementary
(
x = {
var in, out, chain_L, chain_R, chain, numTeeth, phaseOffset, hop = 1/2;
in = PinkNoise.ar(0.5);
// in = Saw.ar(100);
chain_L = FFT(LocalBuf(2048), in, hop: hop);
// for different processing of one buffer we must copy
// as PV UGens are writing to their buffers
chain_R = PV_Copy(chain_L, LocalBuf(2048));
numTeeth = MouseX.kr(1, 20);
phaseOffset = MouseY.kr(0, 1);
// with phase offset
chain = PV_RectComb([chain_L, chain_R], numTeeth, phase: [0, 0.5] + phaseOffset);
IFFT(chain) * hop;
}.play(outbus: ~outOffset);
)
x.release
// Field for experimentation: band splitting leads to a
// large variety of new spatialization ideas:
// different splittings, their changes over time,
// combination with other spatialization concepts ...
// With such strategies, there is a close connection between
// the perception of spatial and timbral changes
// (which is the case anyway, regarding head-related transfer functions / HRTFs)
////////////////////////////////////////////////////////////////////
// Ex.0b
// PV_BinRange (miSCellaneous_lib) for splitting into FFT bins
// based on PV_BrickWall, but instead of wipe parameters (between -1 and +1),
// it takes two bin numbers
(
f = { |loFreq = 800, hiFreq = 1500, fundFreq = 50, amp = 0.1|
var bufSize = 1024, binRange, loBin, hiBin, sig, chain;
sig = Saw.ar(fundFreq, amp);
binRange = s.sampleRate / bufSize;
loBin = (loFreq / binRange).round;
hiBin = (hiFreq / binRange).round;
chain = FFT(LocalBuf(bufSize), sig);
chain = PV_BinRange(chain, loBin, hiBin);
IFFT(chain) ! 2;
};
x = f.play(outbus: ~outOffset);
s.freqscope;
)
x.set(\loFreq, 300);
x.set(\hiFreq, 1000);
x.release;
// for multichannel expansion, an array of mono buffers must be provided
(
g = { |loFreq = #[500, 500], hiFreq = 1500, fundFreq = 50, amp = 0.1|
var bufSize = 1024, binRange, loBin, hiBin, sig, chain;
sig = Saw.ar(fundFreq, amp);
binRange = s.sampleRate / bufSize;
loBin = (loFreq / binRange).round;
hiBin = (hiFreq / binRange).round;
chain = FFT({ LocalBuf(bufSize) } ! 2, sig);
chain = PV_BinRange(chain, loBin, hiBin);
IFFT(chain);
};
x = g.play(outbus: ~outOffset);
s.freqscope;
)
x.set(\loFreq, [200, 200]);
x.set(\loFreq, [300, 1200]);
x.set(\loFreq, [1200, 300]);
x.set(\hiFreq, 2000);
x.release;
////////////////////////////////////////////////////////////////////
// Ex.1
// Spatialisation with band-splitting and MS encoding
// ideas:
// .) spread stereo over n speakers
// .) position neighbour bands dynamically like a fan (but don't separate them totally)
// .) keep lower frequencies rather in the mid (less directional perception anyway)
// helper functions for splitting the spectrum
// problem:
// FFT divides the spectrum into equal frequency ranges (bins),
// depending on the window size:
// range(bin) = fsamp / windowSize
// Obviously, the intervals of the bins become much smaller in the higher registers
// here we divide the spectrum into equal intervals and round to the nearest bins
// alternative solutions:
// divide according to the bark or mel scales (resp. blocks from them):
// https://en.wikipedia.org/wiki/Bark_scale
// https://en.wikipedia.org/wiki/Mel_scale
(
// splitting into equal intervals
// outputs array of minbin and maxbin indices
~splitBands = { |bandNum = 10, sampleRate = 44100, windowSize = 512, minFreq = 16|
var binSize = sampleRate / windowSize, nyquist = sampleRate / 2,
factor, minBins, maxBins, binFreqs;
// divide whole range (interval) into bandNum equal intervals
factor = (nyquist / minFreq) ** (1 / bandNum);
binFreqs = factor ** (1..bandNum-1) * minFreq;
minBins = [0] ++ ((binFreqs / binSize).round);
(minBins.asSet.size != minBins.size).if {
Error("empty band detected, reduce bandNum").throw
};
maxBins = minBins.drop(1) - 1 ++ [windowSize / 2 - 1];
[minBins, maxBins].asInteger
};
~splitFreqs = { |bandNum = 10, sampleRate = 44100, windowSize = 512, minFreq = 16|
var nyquist = sampleRate / 2, factor;
factor = (nyquist / minFreq) ** (1 / bandNum);
factor ** (1..bandNum-1) * minFreq;
};
// shows interpolation from lo to hi frequencies from mid to side
~iplFunc = { |num, lo, hi, curve| { |i| (i / (num - 1)) ** curve * (hi - lo) + lo } ! num };
)
// example
~splitBands.(4, windowSize: 2048)
-> [ [ 0, 5, 28, 168 ], [ 4, 27, 167, 1023 ] ]
band 0 from bin 0 to bin 4
band 1 from bin 5 to bin 27
band 2 from bin 28 to bin 167
band 3 from bin 168 to bin 1023
~iplFunc.(4, 0, 1, 1/3).plot
(
// band indices have to be precalculated for use in spat synthdef
~bufSize = 2048;
~bandNum = 10;
~bandIndices = ~splitBands.(~bandNum, s.sampleRate, ~bufSize, 16);
~abus = Bus.audio(s, 1);
// this SynthDef takes advantage of SC's multichannel expansion and is partially difficult to understand
SynthDef(\bandSplitter_8ch, { |outBus, inBus, lfMidSideWeight = 0,
hfMidSideWeight = 1, midSideWeightCurve = 1, sideAz = 0.4,
frontBackWeight = 0.5, frontWidth = 2, backWidth = 2, sideWidth = 2,
// orientation 0.5 assumes no front speaker
orientation = 0.5, amp = 1|
var in = In.ar(inBus, 2), left, right, mid, side, newLeft, newRight,
newMid, weightedMids, midAtSide, front, back, frontBack,
chains, midSideWeight;
// nested multichannel FFT chain: 10 FFT chains for left and right !
chains = { |i|
{ |j|
// PV_BinRange is a FFT bandpass from miSCellaneous lib
PV_BinRange(
FFT(LocalBuf(~bufSize), in[i], 0.5),
~bandIndices[0][j],
~bandIndices[1][j]
)
} ! ~bandNum
} ! 2;
// arrays of size ~bandNum:
left = IFFT(chains[0]); // 10 band audio for left
right = IFFT(chains[1]); // 10 band audio for right
mid = left + right; // 10 mid signals
side = left - right; // 10 side signals
// midSideWeight = ~iplFunc.(~bandNum, lfMidSideWeight, hfMidSideWeight, midSideWeightCurve);
// need weight for every band
midSideWeight = DC.ar((0..~bandNum-1)).lincurve(0, ~bandNum-1, lfMidSideWeight, hfMidSideWeight, midSideWeightCurve);
// this is difficult !
// for Pan2's pos arg the weight interval [0, 1] muist be mapped to [-1, 1]
weightedMids = Pan2.ar(mid, midSideWeight * -2 + 1).flop; // flop turns 10 x 2 into 2 x 10 array
newMid = weightedMids[0];
midAtSide = weightedMids[1]; // 10 delocated mids used for LR re-calculation
// recalculate L and R for each band, based on newly weighted mid data
newLeft = midAtSide + side * 0.5;
newRight = midAtSide - side * 0.5;
// also weight between front and back
frontBack = Pan2.ar(newMid, frontBackWeight * -2 + 1).flop;
front = frontBack[0];
back = frontBack[1];
// each PanAz is an Array of ~bandNum Arrays of size 8, thus sum it to 8 channels
Out.ar(outBus, PanAz.ar(8, front, 0, 1, frontWidth, orientation).sum * amp);
Out.ar(outBus, PanAz.ar(8, back, 1, 1, backWidth, orientation).sum * amp);
Out.ar(outBus, PanAz.ar(8, newLeft, sideAz.neg, 1, sideWidth, orientation).sum * amp);
Out.ar(outBus, PanAz.ar(8, newRight, sideAz, 1, sideWidth, orientation).sum * amp);
}, metadata: (
specs: (
outBus: ~outOffset,
inBus: ~abus.index,
lfMidSideWeight: [0, 1, \lin, 0, 1],
hfMidSideWeight: [0, 1, \lin, 0, 0],
midSideWeightCurve: [0.1, 10, \lin, 0, 5],
sideAz: [0.1, 0.9, \lin, 0, 0.5],
frontBackWeight: [0, 1, \lin, 0, 1],
frontWidth: [1, 6, \lin, 0, 2],
backWidth: [1, 6, \lin, 0, 2],
sideWidth: [1, 6, \lin, 0, 2],
orientation: [-2, 2, \lin, 0.5, 0.5],
amp: [0, 1, \db, 0, 1]
)
)
).add;
)
// check with scope
s.scope(8)
// start spat fx first (green button)
\bandSplitter_8ch.sVarGui.gui(sliderWidth: 350, labelWidth: 120)
// start source, base case: sides raise with frequency
~src = { Out.ar(~abus, Saw.ar(LFDNoise3.ar(3).exprange(20, 1500) * [1, 1.01], 0.1)) }.play
~src.free;
// decorrelated noise with bandpass
(
~src = {
Out.ar(
~abus,
BPF.ar(
{ PinkNoise.ar() } ! 2,
LFDNoise3.ar(10).exprange(20, 1500),
0.5
)
)
}.play
)
~src.free;
// LR movement
~src = { Out.ar(~abus, Pan2.ar(PinkNoise.ar(0.3), LFDNoise3.ar(10))) }.play
~src.free;
// LR movement of moving filter
(
~src = {
Out.ar(
~abus,
Pan2.ar(
BPF.ar(
PinkNoise.ar(),
LFDNoise3.ar(10).exprange(20, 1500)
),
LFDNoise3.ar(10)
)
)
}.play
)
~src.free;
// stop spat fx (red button)
3 Likes
Along the lines of @jordan’s #2 and similar to @dkmayer’s use of the FFT, one could design a set of complementary bandpass filter kernels using SignalBox’s Signal: *gaussianBank.
This approach will be somewhat expensive, but if you’re will to spend the cost, you’ll get quite a bit of control as to where the individual bands are panned.
Thanks for this inspirational document @dkmayer! Materials of the course are always beloved resources!
The concept of splitting and summing the signal via spatialization according to its sonic content is also a crucial element of Weiss/Weisslich 27 by Peter Ablinger and Thomas Musil, see Weiss/Weisslich 27 which is a very nice piece to experience - although the piece plays with the concept in a constructional manner.
1 Like
Another way is to use the method pvcalc on seperate L/R branches of your FFT chain and multiply the levels of every frequency bin independently to create the panning.
Here’s a stereo example modified from the help file - see https://doc.sccode.org/Classes/PV_ChainUGen.html:
// a sound file
c.free; c = Buffer.read(s, Platform.resourceDir +/+ "sounds/a11wlk01.wav");
// v1 fixed positions
(
{
var in, chain, chainL, chainR, lfo, v;
in = PlayBuf.ar(1, c, BufRateScale.kr(c), loop: 1);
chain = FFT(LocalBuf(1024), in);
chainL = chain.pvcalc(1024, {|mags, phases|
var newMags = mags.collect({arg item, i;
sin(2pi * i / 50) * item;
});
[newMags, phases];
}, frombin: 1, tobin: 250, zeroothers: 0);
chainR = chain.pvcalc(1024, {|mags, phases|
var newMags = mags.collect({arg item, i;
cos(2pi * i / 50) * item;
});
[newMags, phases];
}, frombin: 1, tobin: 250, zeroothers: 0);
0.5 * [IFFT(chainL), IFFT(chainR)];
}.play
)
// v2 - moving pans - higher CPU cost
(
{
var in, chain, chainL, chainR, lfo, v;
in = PlayBuf.ar(1, c, BufRateScale.kr(c), loop: 1);
chain = FFT(LocalBuf(1024), in);
lfo = LFSaw.kr(0.2).range(0,2);
chainL = chain.pvcalc(1024, {|mags, phases|
var newMags = mags.collect({arg item, i;
// sin(2pi * (lfo + ( i / 50))) * item;
(lfo + ( i / 50)).fold(0,1) * item;
});
[newMags, phases];
}, frombin: 1, tobin: 250, zeroothers: 0);
chainR = chain.pvcalc(1024, {|mags, phases|
var newMags = mags.collect({arg item, i;
// cos(2pi * (lfo + ( i / 50))) * item;
(1 - (lfo + ( i / 50)).fold(0,1)) * item;
});
[newMags, phases];
}, frombin: 1, tobin: 250, zeroothers: 0);
0.5 * [IFFT(chainL), IFFT(chainR)];
}.play
)
1 Like
I wrote this (based on my understanding of the language), it is costly in terms of cpu but interesting results.
c = Buffer.read(s, Platform.resourceDir +/+ "sounds/a11wlk01.wav");
(
{
var in, chain, v,b=0;
in = PlayBuf.ar(1, c, BufRateScale.kr(c), loop: 1);
250.do{ chain = FFT(LocalBuf(1024), in);
chain = chain.pvcollect(1024, {|mag, phase, index|
[mag, phase];
}, frombin: b, tobin: b, zeroothers: 1);
b=b+1;
Out.ar(0,PanAz.ar(2,IFFT(chain),LFSaw.ar(0.999.rand)))
}
}.play
)
1 Like