Monthly Archives: January 2017

Plan for the year 2017

In no specific order:


The goal here is to be able to read Chinese web pages and maybe newspapers. This year being able to read simple children’s books seems like a reasonable target. After that, I’ll have to readjust the plan depending on how difficult it seems to be.

At the moment I recognize about 100 characters, know some really simple phrases, can’t read even the most simple texts. Someone said somewhere that knowing 2000 characters (+ of course the grammar etc.) is enough to read and understand most newspapers. So a long, long way to go…

Music production

The main goal is to create enough (decent!) music to make an hour long album/mix out of it. Maybe even play that set somewhere.

Currently I have one song I’m willing to call finished. The good thing about that one song is that in the process of making it I finally got rid of the feeling of maybe not being talented enough or whatever. Anyway, here’s that one song:

It’s not great, but I think it’s good enough!

Of course I have tons of unfinished songs and parts of songs lying around… Maybe they’ll be useful someday.

Musical skills

The goal is to be able to play melodies by ear. Be it on piano or guitar. Maybe even singing?

At the moment after hearing a couple of chords from any major key and then a note in that key, I can tell with 90% accuracy which interval from the root note that note was. So not great, but not completely hopeless either. I’ve been using this app for practicing. I think (hope) as long as I keep practicing every day, I’ll be pretty good way before 2018.

With a guitar in my hand I can very slowly figure out melodies, but it’s painfully slow, and to be honest a bit embarrassing considering how long I’ve been playing the guitar.


The plan is to build and finish a simple synthesizer. Then use at least a single sound out of it in a song.

I’ll have to brush up my electronics skills before I even get to the building part. Currently I have a decent idea of the physics side of electronics and some hands-on experience.

I’m now reading Practical Electronics for Inventors and I’m something like 1/10 through it. I think after I’ve read most of the book and refreshed my memory wrt. soldering etc. I’ll be good to go to improve just by reading and building stuff I find from the interwebs.

I’ve already bought all the components for the synth and they’re sitting at our hacking space near my home. Just add water…

Real-time graphics demo / VR application

This is the most open-ended of these. I’m not sure what I want to do, I just finally want to finish something in this space since I have way too many almost-there type of projects lying around. I’ll hopefully make a new post about what this will be during the first half of the year.

B.Sc in Computer Science

Major in CS, minor in theoretical physics. 16 out of 25 ECTS of theoretical physics already done, about 137/180 of CS done (plus getting 10 ECTS due to work experience is just one email away). The missing 33 ECTS or so should be very much doable in a year, as long as I don’t pick anything overboard as the subject for my bachelor’s thesis. We’ll see about that.


A total of 6 goals for the year of 2017. Meaning I could dedicate 2 months of time to each of them if I don’t do them in parallel. Doesn’t sound easy at all, doesn’t sound impossible. If I get all of these done, the 2018 version of me sounds like an awesome dude.

Procedural Mesh Generation from Live Audio in Unreal Engine 4


Here are two example videos of it running live where at the beginning I just press play in Foobar on Windows, and the visualization begins. Spotify, Youtube, or any other audio source on Windows works straight away as well.

A word of warning though, it’s only a proof of concept, so it is not stable! In-editor it generally works well, but sometimes crashes. The program also does not release all the memory that it should. Due to a yet unknown reason the standalone version basically doesn’t work at all.

Thanks to other open source projects

For the mesh generation part I got a lot of help from SiggiG’s procedural UE4 project/tutorial which lives here:

Some of the code for figuring out frequencies from audio chunks is from eXi’s sound visualization plugin, especially the original CalculateFrequencySpectrum function that can be found here, and his use of the library KissFFT.

Because I’m proud that I was able to figure out the frequency calculating part myself as well, I want to add that I have my own implementation for it (with the help of the library “ffft”), but for this project I overwrote that with eXi’s solution to rule out bugs in that area.

A very brief and dense overview on how it works

On Windows, we’re in luck because we’ve already done the audio capture part, now we just direct it towards UE4 instead of a file. An audio sink for the audio capturer receives chunks of audio frames from the capturer, and the audio listener itself is ran in it’s own thread within the visualizer process. Now that chunks of audio are popping into a queue from which we can dequeue them in the UE4 main thread, we can calculate the sound spectrums for each audio chunk we receive. Finally on each game tick we fetch a list of new frequencies, and if found, add those to the mesh, and move the camera forward to keep up.

Feel free to ask me on Twitter or in the comments if something is unclear!

Making it work on platforms other than Windows

On Linux capturing the audio should be very easy for example by directing arecord ‘s standard output to the UE4 program’s standard input and going forward from there, but haven’t gotten around to trying that yet. On OSX I would start searching for a solution with the help of the project “Soundflower”, but I’m not sure how easy that will be.


Maybe the proof of concept gives someone an idea for something awesome. Please make a new Audiosurf that takes in live audio and doesn’t need to process the whole song from a file first. Or make the sound waves collideable and get a some sort of game mechanic out of that?