ljud.org

Reverse-engineering human rubato

2016-03-22T00:00:00Z

This is a part of a series of posts where I'm trying to explain some of the decisions made in Timmer, a solo project for double-bass and electronics.

Already when recording the source material for Timmer, I knew that the material would be cut up and re-assembled. A lot of the material came to be quite sound-oriented and rhytmical, but often without a fixed tempo. So for one of the recorded tracks, I wanted to create a randomized stream of rhythmic events, based on the recording.

The code is SuperCollider, and the full example outlined below is available on GitHub.

The first step would be to get the durations of the notes. I chose these first 18 onsets to represent a kind of rhythmic motif, from which I would extract some more or less statistically insignificant data.

As with all the other recordings, I did a combination of onset extraction and manual corrections in Sonic Visualiser to make a simple textfile of onset times and durations. The file was imported into SuperCollider, where I started with plotting the durations.

Very exciting. Here we can see that the durations are roughly between 0.5 and 1.5, with a mean somewhere around 0.8. What can we do with this information? Let's make a test instrument and a test pattern to try it out.

Different randoms

~motif = q.player.data.onset[..17].flop[1]; //First 18 onsets of recording
SynthDef(\testDrum, { |freq, amp =1, sustain=0.1|
  var snd = BPF.ar(Decay.ar(Impulse.ar(0), 0.1) * 10, 60, 0.01) * EnvGen.ar(Env.perc(0, sustain), doneAction:2);
  Out.ar(0, (snd * 6 * amp ).softclip.dup);
}).add;

Pbindef(\test_08_Pwhite,
  \instrument, \testDrum,
  \amp, 1,
  \legato, 0.1,
  \dur, Pwhite(~motif.minItem, ~motif.maxItem),
).play;

Here, I'm simply trying to generate random durations between the minimum and maximum duration of the motif, which we soon realize is too random to be musical. Another try:

var maxChange = ~motif.differentiate.maxItem;
Pbindef(\test_08_Pbrown,
    \instrument, \testDrum,
    \amp, 1,
    \legato, 0.1,
    \dur, Pbrown(~motif.minItem, ~motif.maxItem, maxChange)
).play;

Switching the completely-random Pwhite to a Pbrown, which generates a brownian motion, makes the rhythm a bit more musical, but it still doesn't have the same vibe as the original recording. To get closer, we can use the duration data to create a stream of weighted random numbers:

var q = 0.125; //quantize
var weights = ~motif.round(q).asBag.asWeights;
var rand = q * [-0.5, 0.5]; //randomness

Pbindef(\test_08_Pwrand,
    \instrument, \testDrum,
    \amp, 1,
    \legato, 0.1,
    \dur, Pwrand(weights[0], weights[1], inf) + Pwhite(*rand);
).play

Here, I'm rounding the original durations to 0.125, which would correspond to 1/8 in 60 bpm. The durations are counted (through converting the array to a Bag, and converted to an array with durations and weights. This data is passed to a Pwrand, which picks a duration with a probability according to the weights. To keep the rubato feel, I add some randomness with the last Pwhite.

Repeating once or twice

This starts to resemble the original a bit more, so we continue with some other aspects of the recording. Looking at the plot, we can see that some duration values seem to repeat once or twice. Let's do that with our random stream as well.

Pdef(\test_08_Pwrand_stutter, Psmartstutter(Prout({ |ev|
    //Duration threshold. If dur <= 0.05, count it as a repetition
    var repetitionThreshold = 0.05;
    //Make a list of booleans. Is this a repetition?
    var repList = ~motif.differentiate.abs.collect(_ <= repetitionThreshold);
    //Number of non-repeated values
    var uniqueCount = repList.reject(_.value);

    //Repetition choices, starting at a single event, then event + rep, then event + 2x rep...
    var choices = (1..10);
    //Assuming max number of repetitions is 10
    var weights = 0 ! choices.size;

    //Go through every boolean in repList
    var repCount = 0;
    repList.do { |isRep|
        if (isRep) {
            //For every repetition, increment repCount
            repCount = repCount + 1;
        } {
            //For every non-repetition, add repCount to repProb and reset repCount
            weights[repCount] = weights[repCount] + 1;
            repCount = 0;
        }

    };

    //Normalized probability of 0->10 repeats
    weights = weights.normalizeSum;

    loop {
        //According to the plot, only durations between ~0.5 and ~1 is repeated.
        if (ev.dur > 0.45 and: { ev.dur < 1.05 }) {
            //Repeat each event x times

            choices.wchoose(weights).yield;
        } {
            1.yield;
        }

    }

}), Pdef(\test_08_Pwrand))
).play;

First we extract the probability of a duration being played n times from the recording, where n is a number from 1 to 10. Then we choose a value if the duration of the current event is between 0.45 and 1.05, meaning that short or long notes never repeat. Sounds fun.

Grace notes

Another recurrent motif in the motif, are grace notes. Listening to the track, it seems that they come with the longer notes, and after some counting, we can see that 22% of all notes have grace notes. Then, what is the probability for a grace note among only the longer notes? Some math later, we come up with an answer that sounds all right. If note is long, and coin toss is true, we output two notes with a random (0.08-0.15) strum value, which separates the notes.

var middle = [~motif.minItem, ~motif.maxItem].mean;

//Let prob be probability of a note longer than mean
var prob = ~motif.reject(_ < middle).size / ~motif.size;

//And if note is longer than mean, what should the probability be
//to make the total probability of a grace note be 0.22?
prob = 0.22/prob; //~0.66

Pdef(\test_08_Pwrand_stutter_grace, Pchain(
    Pbind(

        \amp, Pfunc { |ev|

            var out = 0.8.rrand(1); //some random amp

            //If this test passes, output an array, creating two notes
            //Assuming Pbrown has an even distribution, which it hasn't
            if (ev.dur > middle and: { prob.coin }) {
                out = [0.1.rrand(0.2), out];

                if (0.5.coin) { out = out.reverse }

            };
            out
        },
        \strum, Pwhite(0.08, 0.15)
    ),
    Pdef(\test_08_Pwrand_stutter)
)).play;

If Then Legato

In the excerpt, some notes are longer than others. They also seem to come in pairs. Two times during the short track we go into a 'branch' of long, short note pairs. Here we translate that into a 2/18 probability of entering such a branch, and a 90% chance of repeating the pair twice (even though it's 100% in the excerpt).

Pdef(\\test_08_Pwrand_stutter_grace_legato, Pchain(
    Pbind(
        \\sustain, p { |ev|
            var prob = 2/~motif.size;
            var sustainShort=0.1;
            var sustainLong=0.5;
            loop {
                //If we have legato, we go into this branch
                if (prob.coin) {
                    //90% chance of repeating twice
                    [1,2].wchoose([0.1, 0.9]).do {
                        sustainLong.yield;
                        sustainShort.yield;
                    }
                };
                //Always yield short after a group of longs
                sustainShort.yield;
            }
        },
        //Adjust grace notes
        \\sustain, Pfunc { |ev|
            if (ev.dur.isArray) {
                ev.sustain = [0.1, ev.sustain];
            };
            ev.sustain
        }
    ),
    Pdef(\\test_08_Pwrand_stutter_grace)
).trace).play;

Why?

Now we have something which might be described as the Frankenstein version of the excerpt above. Note that we haven't really worked on the timbre, dynamics, or other important parameters, and that a lot of fine-tuning could be done to make the rhythm stream closer to the original. But still, this might be usable in some way, which leads us to the question in the headline: Why do this?

One goal with Timmer is to find ways of extracting and extrapolating musical information from the solo recordings, to use that information as a base for generative processes. I can imagine taking this data and develop it further, to make a composition based on the seed of rhythmic ideas expressed in this excerpt.

Going through this analytic process forces you to listen in a different way. When you start thinking about musical events in terms of probabilities it makes you loosen up the linear way of thinking about musical structures, which can be a great way to expand your music-making toolset.

When writing this, the track is not finished, but I will continue working with this material. The main challenge, for this track and the whole project, is to make something that integrates with and extends the original recording. Let's see how it goes.

An update

2015-11-16T00:00:00Z

Timmer is the working title for a solo project involving double bass and electronics, that has been going on (and off) for the past several years. This autumn I'm fortunate enough to be able to work on it full time for a couple of months. I will try to log the progress here, and very soon also put up some sounds. But first, a long introduction, crammed with all those technical details you love to hear about.

I'm currently preparing the electronics setup, based on an old idea documented here. I'm using transducer speakers mounted on the bass to create a feedback loop, together with the regular contact mic and some filtering in SuperCollider. The filtering and levels should be balanced so the feedback is only happening when a string is open. This makes it possible to manipulate the physical properties of the instrument, making strings resonate infinitely and other nice things.

The idea works in theory, but the hard part has been to find a simple way to do the filtering, which keeps as much of the signal as possible without breaking anything (ears, speakers or bass). Also, I've had problems trying to balance the amplitude of different notes.

The first draft of the instrument used parallel bandpass filters, tuned in semitones. To balance the amplitude of the notes, the gain could be adjusted, and I also experimented (a lot) with setting individual short delays on each filter, to adjust the phase. This took a lot of time, but wasn't very effective. I never really understood what was happening and why some notes were so much louder than the others.

I started to experiment a bit with FFT filtering, trying out UGens like PV_LocalMax and PV_MaxMagN, to only let the most important partials through. I found a few nice effects, but the main problem was still there, with uneven response for different notes.

So, I put a compressor in the end of the filtering chain, which really made a difference. By accident I also bypassed the FFT filtering, which left me with just a lowpass filter, and a couple of peak filters which was supposed to even out the frequency response of the mic and bass. This was the best version yet, and also the simplest. I later expanded it to use 24 peak filters, tuned in semitones from C1-B2. Or was it 25? Something like that.

So, now we have a double bass, a volume pedal, and possible infinite sustain. What can you do with this, besides making drones? Probably a good topic for the next post.

TV4 Avslutning 1992-02-18

2011-09-14T00:00:00Z

http://www.youtube.com/v/KD611_1NATs

First start after the winter

2010-05-06T00:00:00Z

http://www.youtube.com/watch?v=lcbea2wKHJo

Dominique Young Unique vs Alfredo Sanchez – War Talk

2010-05-03T00:00:00Z

http://soundcloud.com/alfredo-sanchez/war-talk-alfredo-sanchez-remix

Lose/Lose

2010-04-13T00:00:00Z

stfj 3.0

Lose/Lose is a video-game with real life consequences. Each alien in the game is created based on a random file on the players computer. If the player kills the alien, the file it is based on is deleted. If the players ship is destroyed, the application itself is deleted.

Untitled

2010-04-06T00:00:00Z

YouTube - Insane 4000 BMP Double Pedal

OF COURSE this double pedal work is sped up. Geez, I'm not Mr.Lech or George Kollias.

Crystal

2010-03-23T00:00:00Z

CRYSTALのホームページへようこそ

Im Krankenhaus

2010-03-16T00:00:00Z

Dr. Pravda’s Ultimate Amiga Rave

Dr. Pravda – one of the most blunt Amiga music on record. No fancy tricks or subtleties, this is pure unadulterated tracker rave!

PantyChrist Sunday

2010-03-14T00:00:00Z

PantyChrist (x2) | Spisa

Noise meets Noise meets Cabaret
Tokyo meets San Francisco meets NYC meets Venus
Godzilla meets John Cage meets Princess Grace