Physical modeling of an acoustic guitar.
category: code [glöplog]
@BarZoule in a physical model, the propagation of sound waves in a medium (a string, a column of air or other 2D/3D surfaces) is modeled. Several methods exist.
The most well-known model for a string, for example, is the Karplus-Strong delay-line based, aka 'waveguide' model. Julius Orion Smith III has a good web page on it.
A second (more expensive) method is using finite difference equations. Stefan Bilbao (use google) has done lots of work in this direction.
Optimizations and shortcuts usually introduce inaccuracies into the model, but every model is inaccurate to some extent. Whether or not you want to call your model a 'physical model' is mostly up to the designer. As you might suspect, commercial audio products tend to call their synthesis methods 'physically modeled' or 'virtual' mostly for marketing (read: bullshit) reasons.
The most well-known model for a string, for example, is the Karplus-Strong delay-line based, aka 'waveguide' model. Julius Orion Smith III has a good web page on it.
A second (more expensive) method is using finite difference equations. Stefan Bilbao (use google) has done lots of work in this direction.
Optimizations and shortcuts usually introduce inaccuracies into the model, but every model is inaccurate to some extent. Whether or not you want to call your model a 'physical model' is mostly up to the designer. As you might suspect, commercial audio products tend to call their synthesis methods 'physically modeled' or 'virtual' mostly for marketing (read: bullshit) reasons.
trc_wm: Just read the introduction to Bilbaos book, seems pretty interesting, thanks for the hint.
I'm not sure if it helps, but here's an image of a physical model of an acoustic guitar:
Tnx! It's much clearer now!
As Bird said, FTM_STRING is afaik one of the best.
The fact that I bugfixed that code with Stefan, is of course a mere coincidence :) (well, I haven't tried the link Bird's gave - if the CPU time raises to 100% few seconds after the sound has vanished, it's probably still the unfixed version)
The fact that I bugfixed that code with Stefan, is of course a mere coincidence :) (well, I haven't tried the link Bird's gave - if the CPU time raises to 100% few seconds after the sound has vanished, it's probably still the unfixed version)
Sound clip, or it didn't happen.
Most of those karplus strong sims seem to all get the same things wrong:
- A real string instrument has a LOT of beating going on, at all sorts of weird frequencies on each harmonic. In other words, 2x parallel delays/waveguides is the minimum here, and you absolutely need to have multiple allpass filters in one of the feedback loops (to detune the harmonics in all sorts of weird ways so that they beat at different frequencies). Pianos have even more beating all over the place and you will not get good results unless you take that into account.
- Way to much filtering! You have to be really subtle on the filter, don't eat up all the high frequencies! (I'm thinking a shelf or 1 pole filter might be better than the usual 2 pole here - potentially for both the impulse and especially for the feedback loop)
- A real string instrument has a LOT of beating going on, at all sorts of weird frequencies on each harmonic. In other words, 2x parallel delays/waveguides is the minimum here, and you absolutely need to have multiple allpass filters in one of the feedback loops (to detune the harmonics in all sorts of weird ways so that they beat at different frequencies). Pianos have even more beating all over the place and you will not get good results unless you take that into account.
- Way to much filtering! You have to be really subtle on the filter, don't eat up all the high frequencies! (I'm thinking a shelf or 1 pole filter might be better than the usual 2 pole here - potentially for both the impulse and especially for the feedback loop)
madbrain: Let me pick your brain. When you say 2x delays/waveguides, do you mean these should be of similar feedback / both/all be part of the same string? Or do you mean that the additional waveguides should be used to simulate secondary feedback effects like the body of the instrument?
In the early 2000, I did some work to generate a guitar sample instead of storing it. Sadly, I could not find the work I did, which was not very advanced anyway.
But, here is what I did back then... I asked a friend of mine to give me a guitar sample (since I do not have guitar).
I use matlab, load the sample and conduct a serie of FFT of size N samples at every N/M (where M>2). I look for the fundamental frequency and their harmonics. In total having L frequencies. Then, for each given frequency, I took the FFT results in amplitude in time which give me a new vector.
This vector contain the envelope which corresponds to the main variation. I remove the envelope using a differential between the signal and a generated envelope which used 2 exponential. One exponential for attack and one for decay.
On the remainder, I conduct again FFT with overlap like at my first step. In order to get amplitudes and frequencies which corresponds (I though) of the variation of energy that transfer from one harmonic to another or from kinetic to potential energy.
I then obtain a formula which looked like this:
Amplitudes:
Envelope-per-harmonic:
Ae(x=1..N) = {t<d(x) | exp(t, attack(x))} | {t>=d(x) | 1.0-exp(t, decay(x))}
Amplitude and Frequency per amplitude of harmonic:
A(x=1..N) = Ae(x) * [sum(1..M) a(m,x) * sin( f(m,x) )]
The generated signal to reconstruct the sample was:
S = sum(1..N) A(x) * sin ( f * x )
Where each harmonic has their own attack-decay time and exponent combine with a serie of sine wave with their own frequency and amplitude.
These coefficient was enough to construct a sample with an error from the original signal very low (I don't remember how low the error coefficient was). But, the real test I was looking is to burn a CD (yep, back then, I burn a CD to try my results.. not too much user friendly :-P) and put into a descent stereo... I place the real and the reconstruct signal a couple of times and make it play in loop. I was not able to know which one is which.
My next step.. would have been to ask other samples and do the same analysis... and when I would have the tables for various frequency... I may wanted another guitar.. or to analyze how the coefficient vary in frequency and type of string.
There was already some kind of patterns in the data I obtain. But, since I was using only 1 sample.. did that pattern was really meaningful? Or maybe I just happen to have a special case.
But, here is what I did back then... I asked a friend of mine to give me a guitar sample (since I do not have guitar).
I use matlab, load the sample and conduct a serie of FFT of size N samples at every N/M (where M>2). I look for the fundamental frequency and their harmonics. In total having L frequencies. Then, for each given frequency, I took the FFT results in amplitude in time which give me a new vector.
This vector contain the envelope which corresponds to the main variation. I remove the envelope using a differential between the signal and a generated envelope which used 2 exponential. One exponential for attack and one for decay.
On the remainder, I conduct again FFT with overlap like at my first step. In order to get amplitudes and frequencies which corresponds (I though) of the variation of energy that transfer from one harmonic to another or from kinetic to potential energy.
I then obtain a formula which looked like this:
Amplitudes:
Envelope-per-harmonic:
Ae(x=1..N) = {t<d(x) | exp(t, attack(x))} | {t>=d(x) | 1.0-exp(t, decay(x))}
Amplitude and Frequency per amplitude of harmonic:
A(x=1..N) = Ae(x) * [sum(1..M) a(m,x) * sin( f(m,x) )]
The generated signal to reconstruct the sample was:
S = sum(1..N) A(x) * sin ( f * x )
Where each harmonic has their own attack-decay time and exponent combine with a serie of sine wave with their own frequency and amplitude.
These coefficient was enough to construct a sample with an error from the original signal very low (I don't remember how low the error coefficient was). But, the real test I was looking is to burn a CD (yep, back then, I burn a CD to try my results.. not too much user friendly :-P) and put into a descent stereo... I place the real and the reconstruct signal a couple of times and make it play in loop. I was not able to know which one is which.
My next step.. would have been to ask other samples and do the same analysis... and when I would have the tables for various frequency... I may wanted another guitar.. or to analyze how the coefficient vary in frequency and type of string.
There was already some kind of patterns in the data I obtain. But, since I was using only 1 sample.. did that pattern was really meaningful? Or maybe I just happen to have a special case.
F-C: You're pretty much reconstructing the very same sample using a FFT series. Not very interesting in itself. Interesting would be if you could change the some parameters, say pitch, gate time for the note, maybe simulate note velocity. If you could to that and it still sounded realistic, then you would have a useful synthesis algorithm. Otherwise you only have a very fancy way of playing back a sample.
For example, like madbrain almost mentions above, there are effects that are independent of the string and its pitch, but dependent on the body of the guitar. If you could strike one note then another note with your algorithm, it wouldn't sound like two strings on a guitar, but like a sample that was just pitched.
For example, like madbrain almost mentions above, there are effects that are independent of the string and its pitch, but dependent on the body of the guitar. If you could strike one note then another note with your algorithm, it wouldn't sound like two strings on a guitar, but like a sample that was just pitched.
nitro2k01: I know... but, my first step was to obtain list of data from a real guitar. Further analysis could have give me (hopefully) more clue about the changes and effects.
I admit my work is not practical, but for research it is a starting point (maybe not the best one... either).
But my idea was to instead of doing just a serie of FFT for analysing, it was to iterate FFT from the main simple to each frequency waveform and so on. I though applying a similar concept than when we do derivative computation with 1st, 2nd, 3rd order form.
I admit my work is not practical, but for research it is a starting point (maybe not the best one... either).
But my idea was to instead of doing just a serie of FFT for analysing, it was to iterate FFT from the main simple to each frequency waveform and so on. I though applying a similar concept than when we do derivative computation with 1st, 2nd, 3rd order form.
Nice materials here: http://www.acoustics.hut.fi/~vpv/
nitro2k01 : Well, a string vibrates on at least 4 different planes afaik: horizontal (left-right), vertical (towards-away from the body), "longitudinal" (up-down - ie compression-decompression), torsional (twisting). These planes will often have different sound speeds - horizontal and vertical will only be slightly different, afaik longitudinal is higher, and torsional is a lot higher. In practice, afaik only the horizontal and vertical planes count for much in a guitar. For a piano string, most notes have 3 strings, which is 12 potential vibration planes! (though afaik in a practical simulation you only really need 3)
Then you have bridge/body interaction. Here a given harmonic can interact with one of the body's harmonics, creating yet another opportunity for beating! Or, more likely, the body will detune the horizontal vibration from the vertical one, which will make them beat together. Or, the body can soak up a harmonic from the sound, making it much louder but much shorter.
The hard part is that those beatings aren't very consistent from harmonic to harmonic. One harmonic might beat a lot while the next one might almost not beat at all! Or they can beat at completely different rates, and so on.
Then you have bridge/body interaction. Here a given harmonic can interact with one of the body's harmonics, creating yet another opportunity for beating! Or, more likely, the body will detune the horizontal vibration from the vertical one, which will make them beat together. Or, the body can soak up a harmonic from the sound, making it much louder but much shorter.
The hard part is that those beatings aren't very consistent from harmonic to harmonic. One harmonic might beat a lot while the next one might almost not beat at all! Or they can beat at completely different rates, and so on.
I just wanted to bump this thread because it is awesome. I enjoy reading it, although I have nothing to say. Perhaps it floating to the top will lead to more comments. : D
How about this way:
Six string models (of choice), but to acurately model bridge, into each string output of five others is fed (through carefully set impdance)
then the body (plate resonances and Helmholz resonance of the hollow) can be done by convolution filter (or something like the "final trnser function" in ZynAddSubFx).
Six string models (of choice), but to acurately model bridge, into each string output of five others is fed (through carefully set impdance)
then the body (plate resonances and Helmholz resonance of the hollow) can be done by convolution filter (or something like the "final trnser function" in ZynAddSubFx).
lol @ryg. but right. this obviously the best sounding model. :D
"but to acurately model bridge, into each string output of five others is fed (through carefully set impdance)"
That's easier said than done! The 12 "vibration planes" (2 planes per string) probably feed into each other in a completely convoluted way, probably frequency dependent and with some phase shifts. Just summing together your 6 strings to refeed them probably won't cut it! Good luck getting good measurement data here.
As for the body, I'm not really a fan of the convolution filter (it has latency or cpu usage issues, plus it's hard to couple it with the strings, so I'd do it with modal synthesis or a specially made reverb instead) but yeah, that'd be doable.
That's easier said than done! The 12 "vibration planes" (2 planes per string) probably feed into each other in a completely convoluted way, probably frequency dependent and with some phase shifts. Just summing together your 6 strings to refeed them probably won't cut it! Good luck getting good measurement data here.
As for the body, I'm not really a fan of the convolution filter (it has latency or cpu usage issues, plus it's hard to couple it with the strings, so I'd do it with modal synthesis or a specially made reverb instead) but yeah, that'd be doable.
The body filter is usually assumed to be common for all filters and the strings are modelled by 6 or 12 cross coupled delay lines (DFN) with very simple damping filters in each string. This is the "easy" part of a virtual acoustic guitar. The hard part is modelling all the remaining artifacts such as fret and finger noises, the picking style and the voicings. Without taking these into account, it wil sound lifeless.
on IRTC... (use to be a competition with pov-ray)...
trc_wm: That's more like a wind instrument problem - you can good sounding realistic wind instruments with a simple delay line model but then you have to put in all the expression so you really have to "play" the instrument (usually using monophonic mode and lots of CC wiggling).
Of course, plucked strings also have this problem. But also, all plucked strings I've heard except Pianoteq have the this awful "canny" sound, in varying degrees - simple Karplus has it a lot but even good, modern implementations have it (see Korg Kronos). It's what killed the Yamaha VP-1, too.
The problem is that it makes those simulations sound both fake AND bad - if it sounded fake but good, physical modelling would probably be mainstream! I'm just not sure _what_ this problem is though - wrong/oversimplified inharmonicity or beats or decay rate or general spectrum? (solvable with modal synthesis, but then requires tons of control data and basically turns into additive resynthesis) Wrong attack/transient/phase? (would require a much better pluck model) Lack of nonlinearity in the string? (if it's this, it's _probably_ going to be EXTREMELY hard to solve)
So yeah, I'm starting to think is that there's something fundamental that's missing from the algorithm, something basic that dooms any attempts to failure, but I can't quite tell what. I'd probably try modal synthesis but that's just a hunch.
There's not much point to a physical model if it can't sound at least better than at least a really good FM patch, and unfortunately a lot of karplus strong implementations you'll find can't even beat the old DX7, let alone a sample set!
Of course, plucked strings also have this problem. But also, all plucked strings I've heard except Pianoteq have the this awful "canny" sound, in varying degrees - simple Karplus has it a lot but even good, modern implementations have it (see Korg Kronos). It's what killed the Yamaha VP-1, too.
The problem is that it makes those simulations sound both fake AND bad - if it sounded fake but good, physical modelling would probably be mainstream! I'm just not sure _what_ this problem is though - wrong/oversimplified inharmonicity or beats or decay rate or general spectrum? (solvable with modal synthesis, but then requires tons of control data and basically turns into additive resynthesis) Wrong attack/transient/phase? (would require a much better pluck model) Lack of nonlinearity in the string? (if it's this, it's _probably_ going to be EXTREMELY hard to solve)
So yeah, I'm starting to think is that there's something fundamental that's missing from the algorithm, something basic that dooms any attempts to failure, but I can't quite tell what. I'd probably try modal synthesis but that's just a hunch.
There's not much point to a physical model if it can't sound at least better than at least a really good FM patch, and unfortunately a lot of karplus strong implementations you'll find can't even beat the old DX7, let alone a sample set!
I've got an idea: Let connect all the parameters to the neural network, and then feed the thing with some good guitar recordings backed with score/tabulature, and let the system learn.:)
(Again: my programming skills are not enough to try it out.)
(Again: my programming skills are not enough to try it out.)
if you're going to use machine learning on a problem like this, I'm not sure a neural network would be the best approach (although I know you're just joking).
Dr Claw, on the other hand might have something to say . . .
Dr Claw, on the other hand might have something to say . . .
bizun : I think the Hartmann Neuron had some kind of neural net synthesis. Seemed more of a sound designer synth than a musician synth.
Here is a sound generator string:
http://tuapse-mikle.narod.ru/String.zip
Options:
1. Tension force
2. Length
3. Damping (stub)
4. Roughness
5. The frequency of the Q filter
6. The reciprocal of Q filter.
That's the sound of the bow on the string:
http://tuapse-mikle.narod.ru/Viola.zip
And it is - an experiment varies non-linear string, but the plane:
http://tuapse-mikle.narod.ru/Bell.zip
http://tuapse-mikle.narod.ru/String.zip
Options:
1. Tension force
2. Length
3. Damping (stub)
4. Roughness
5. The frequency of the Q filter
6. The reciprocal of Q filter.
That's the sound of the bow on the string:
http://tuapse-mikle.narod.ru/Viola.zip
And it is - an experiment varies non-linear string, but the plane:
http://tuapse-mikle.narod.ru/Bell.zip
