Phonography / Sonogramme on Raspberry Pi

Pixilang programming language
Post Reply
ainegil
Posts: 171
Joined: Thu Sep 22, 2022 11:37 pm

Phonography / Sonogramme on Raspberry Pi

Post by ainegil »

ainegil
Posts: 171
Joined: Thu Sep 22, 2022 11:37 pm

Re: Phonography / Sonogramme on Raspberry Pi

Post by ainegil »

ainegil
Posts: 171
Joined: Thu Sep 22, 2022 11:37 pm

Re: Phonography / Sonogramme on Raspberry Pi

Post by ainegil »

This one is mor about the music.

Spectrum warped on auditory ERB scale as above. Rendered on Raspberry.
The mirrored upper half was added in KDEnlive.

ainegil
Posts: 171
Joined: Thu Sep 22, 2022 11:37 pm

Re: Phonography / Sonogramme on Raspberry Pi

Post by ainegil »

same track, nicer visuals



There are 4 renders in Pixilang, the ray effect was added afterwards in Alight Motion.

The color and note extraction is done by cepstrum, (spectrum of the spectrum)

Its still not working as intended, cause the low notes are blurred
(that is several neighbouring notes are selected), and there are also notes spaced an octave apart.

FFT size is 4096 iirc, and window size is 2* samplerate/fps
fps is 30 hz.
Window is half sine ( sine from 0 - pi)

The colors are set as repeating rainbow per octave, and and an additional color is picked by the pitch if the strongest peak in cepstrum.

when there is a peak in cepstrum, the bin of the spectrum is calculated and the magnitude of 4 harmonics are added.

The magnitudes have a peak detector ( peak envelope follower) for smoothing.

If s.o. is interested I can post source code, but its not commented well nor optimized.
I just hack till it looks nice.

It does not run full fps in real time but fast enough to preview on Raspberry 400.
I estimate ca. 15 fps.
ainegil
Posts: 171
Joined: Thu Sep 22, 2022 11:37 pm

Re: Phonography / Sonogramme on Raspberry Pi

Post by ainegil »

here I tried to extract motion to visualize melody.

Uses the cepstrum to extract pitches, polyphonic but the tracks were seperated into string and bass without drums.
Then uses the loudest pitch and the dots try to reach the horizontal line of the loudest pitch.
It doesnt work very well due to octave jumps and ambiguities, but I think its still useful.

But I am still trying to improve the analysis.

ainegil
Posts: 171
Joined: Thu Sep 22, 2022 11:37 pm

Re: Phonography / Sonogramme on Raspberry Pi

Post by ainegil »

same script rewritten for different music and different visuals

ainegil
Posts: 171
Joined: Thu Sep 22, 2022 11:37 pm

Re: Phonography / Sonogramme on Raspberry Pi

Post by ainegil »

more beautiful version with a different track



There is a lot of post effects however.

What I noticed and what is different in this version is that the cepstrum pitch estimation seems to work better when the fft is 8196.
Before it hat been 32k because I thought this would be more accurate.

2048 at 44.k sample rate seemed to small however.

Its working a little bit better now.
Post Reply