Author Archives: Teemu Sarapisto

Long time no see

I haven’t forgotten this blog! It’s just that I haven’t felt the need to describe any smaller projects in detail, and the bigger ones seem to still take a while. Still, here’s something:

My first live gig with my own music:

… though that is already a year old. So here’s something quite recent:

also I got this working just today:

I’m also 2 small courses away from my B.Sc, for which I did a B.Sc thesis on Convolutional Neural Nets, and working on my M.Sc. I haven’t stopped learning Chinese either.  Basically everything I was working on 2 years ago is still underway, but maybe in a year or two I’ll finish my studies at least.

I feel like I have a lot to say so maybe a new post in less than a year after this one… We’ll see.

Plan for the year 2017

In no specific order:

Chinese

The goal here is to be able to read Chinese web pages and maybe newspapers. This year being able to read simple children’s books seems like a reasonable target. After that, I’ll have to readjust the plan depending on how difficult it seems to be.

At the moment I recognize about 100 characters, know some really simple phrases, can’t read even the most simple texts. Someone said somewhere that knowing 2000 characters (+ of course the grammar etc.) is enough to read and understand most newspapers. So a long, long way to go…

Music production

The main goal is to create enough (decent!) music to make an hour long album/mix out of it. Maybe even play that set somewhere.

Currently I have one song I’m willing to call finished. The good thing about that one song is that in the process of making it I finally got rid of the feeling of maybe not being talented enough or whatever. Anyway, here’s that one song:

It’s not great, but I think it’s good enough!

Of course I have tons of unfinished songs and parts of songs lying around… Maybe they’ll be useful someday.

Musical skills

The goal is to be able to play melodies by ear. Be it on piano or guitar. Maybe even singing?

At the moment after hearing a couple of chords from any major key and then a note in that key, I can tell with 90% accuracy which interval from the root note that note was. So not great, but not completely hopeless either. I’ve been using this app http://fet.kaizen9.com/ for practicing. I think (hope) as long as I keep practicing every day, I’ll be pretty good way before 2018.

With a guitar in my hand I can very slowly figure out melodies, but it’s painfully slow, and to be honest a bit embarrassing considering how long I’ve been playing the guitar.

Electronics

The plan is to build and finish a simple synthesizer. Then use at least a single sound out of it in a song.

I’ll have to brush up my electronics skills before I even get to the building part. Currently I have a decent idea of the physics side of electronics and some hands-on experience.

I’m now reading Practical Electronics for Inventors and I’m something like 1/10 through it. I think after I’ve read most of the book and refreshed my memory wrt. soldering etc. I’ll be good to go to improve just by reading and building stuff I find from the interwebs.

I’ve already bought all the components for the synth and they’re sitting at our hacking space near my home. Just add water…

Real-time graphics demo / VR application

This is the most open-ended of these. I’m not sure what I want to do, I just finally want to finish something in this space since I have way too many almost-there type of projects lying around. I’ll hopefully make a new post about what this will be during the first half of the year.

B.Sc in Computer Science

Major in CS, minor in theoretical physics. 16 out of 25 ECTS of theoretical physics already done, about 137/180 of CS done (plus getting 10 ECTS due to work experience is just one email away). The missing 33 ECTS or so should be very much doable in a year, as long as I don’t pick anything overboard as the subject for my bachelor’s thesis. We’ll see about that.

Conclusion

A total of 6 goals for the year of 2017. Meaning I could dedicate 2 months of time to each of them if I don’t do them in parallel. Doesn’t sound easy at all, doesn’t sound impossible. If I get all of these done, the 2018 version of me sounds like an awesome dude.

Procedural Mesh Generation from Live Audio in Unreal Engine 4

Source: https://github.com/Tsarpf/UE4-Procedural-Audio

Here are two example videos of it running live where at the beginning I just press play in Foobar on Windows, and the visualization begins. Spotify, Youtube, or any other audio source on Windows works straight away as well.

A word of warning though, it’s only a proof of concept, so it is not stable! In-editor it generally works well, but sometimes crashes. The program also does not release all the memory that it should. Due to a yet unknown reason the standalone version basically doesn’t work at all.

Thanks to other open source projects

For the mesh generation part I got a lot of help from SiggiG’s procedural UE4 project/tutorial which lives here: https://github.com/SiggiG/ProceduralMeshes

Some of the code for figuring out frequencies from audio chunks is from eXi’s sound visualization plugin, especially the original CalculateFrequencySpectrum function that can be found here, and his use of the library KissFFT.

Because I’m proud that I was able to figure out the frequency calculating part myself as well, I want to add that I have my own implementation for it (with the help of the library “ffft”), but for this project I overwrote that with eXi’s solution to rule out bugs in that area.

A very brief and dense overview on how it works

On Windows, we’re in luck because we’ve already done the audio capture part, now we just direct it towards UE4 instead of a file. An audio sink for the audio capturer receives chunks of audio frames from the capturer, and the audio listener itself is ran in it’s own thread within the visualizer process. Now that chunks of audio are popping into a queue from which we can dequeue them in the UE4 main thread, we can calculate the sound spectrums for each audio chunk we receive. Finally on each game tick we fetch a list of new frequencies, and if found, add those to the mesh, and move the camera forward to keep up.

Feel free to ask me on Twitter or in the comments if something is unclear!

Making it work on platforms other than Windows

On Linux capturing the audio should be very easy for example by directing arecord ‘s standard output to the UE4 program’s standard input and going forward from there, but haven’t gotten around to trying that yet. On OSX I would start searching for a solution with the help of the project “Soundflower”, but I’m not sure how easy that will be.

Thanks!

Maybe the proof of concept gives someone an idea for something awesome. Please make a new Audiosurf that takes in live audio and doesn’t need to process the whole song from a file first. Or make the sound waves collideable and get a some sort of game mechanic out of that?

tsatter.com

tsatter.com is what resulted from me wanting to make a website, and take it far enough to call it finished. I learned a lot along the way. Here’s a short description of the whole system. I’ll write more specific posts and link them here when I find the time and interest. Requests for speeding up a specific part are of course welcome.

  • Dockerized environment for easy development and deployment
    • Provided you have docker installed, it takes executing one command to have the whole 5 part infrastructure set up for you.
  • NGINX
    • efficiently shares static files
  • NodeJS
  • IRC server
    • handles actually delivering messages between users in channels, and keeps track of channel’s ops etc.
    • provides a tried and tested protocol for bots to interface with.
  • AngularJS front-end
    • infinite scrolling front page
    • infinite scrolling messages
      • clicking the timestamp gives a shareable url. Opening the URL will take the opener to that message, and highlight it.
      • no need to load the whole history at once
      • handles massive amounts of lines without slowing the computer much
    • infinite scrolling images
      • images have a link to the message in which they were linked
    • inline image search & linking (by pressing @ in a chat)
    • kicking/oping/unoping users in a channel

It used to have registering and logging in as well at a time, but I wanted to make starting using it as easy and low effort as possible. But now I’m thinking maybe sending those post registration spam emails would help with user retention and actually getting users back to it after they’ve tried it once.

Linux productivity

 

I used to install Linux on a whim, and when I couldn’t reach the level of productivity I was used to in Windows (since I had been using it for so many years), I’d go back to Windows in a few weeks. What finally changed this and made Linux my favorite OS to develop in was i3(wm). Here’s a couple other productivity tips:

Vim

  • `vim -p file1 file2 …` opens multiple files in tabs in vim. With zsh expanding regex, `vim -p ./*` opens all files in your current directory as tabs in vim. `vim .`  browses the current folder using vim
  • use `:! <bash command>` like `:! ls` to run normal bash commands from vim and see the results in that window.
  • Use Awesome vimrc‘s awesome version as a base of your .vimrc if you don’t already have one. Also find the super pretty `wombat256mod` color scheme somewhere, put it to ~/.vim/colors/  and add a `set colorscheme wombat256mod` to your ~/.vimrc.
  • After installing the awesome vimrc, add `execute pathogen#infect()` to your .vimrc, then you can install vim plugins just by cloning them from github to `.vim/bundle/`

Window manager / i3

  • Use a tiling window manager, like awesome or i3wm. I’ve never used awesome, because i3 is so awesome, but definitely give one of those a try. They’re actually the killer feature that made me stick with Linux for the first time. Neither Windows or OSX have anything as good as i3.
  • Change i3 directional keybinds to vim keybinds, since they’re by default  one-to-the-right-of-vim-arrows (jkl; instead of hjkl) which confuses the hell out of your brain when you use both vim and i3

General

  • use `xset r rate <time in milliseconds until key repeat starts> <repeats per second>` for example `xset r rate 200 40` to make scrolling around vim faster. The default speed limits are for old people.
  • use zsh and oh-my-zsh rather than bash. It’s so much better (case insensitivity, for one), and easy to use if you’re used to bash.
  • make a folder called dotfiles in your home folder, put the files you’d like to sync between installations there, version control it (I have a private Github repo for it), and symlink them from that folder to your ~/
  • Use `mosh` instead of `ssh` if you often work over high latency connections.
  • If you need to follow multiple log files when executing something in the command line, instead of having multiple terminals for logging or having a different split for each file in the same terminal, you can interleave the logs by opening each log file with `tail -f /tmp/logfile &` to the background. This way as the log entries come in, they’ll be in chronological order in the same shell.

Free-time projects vs. university studies

Something I can’t seem to decide on is whether to spend my spare time on studying or on churning out programming projects to get more actual programming experience, a richer CV, and “lottery tickets” in the sense that one of the projects could some day end up big.

Usually it would be obvious that studies are the way to go, but since I believe VR is going to break through in the next couple of years, and in the long run it’s very likely going to change our lives as much as mobile phones and the internet did and do, I think the sooner you get really deep into it, the better are your chances of knowing just what people are going to want next, and to already have that ready when it happens.

In addition to the big and utopistic things like the metaverse and what have you, which definitely are not things someone is going to code up some night alone, just like with mobile phones where someone made the first torchlight app, there’s going to be a lot of convenience software for VR where early adopters iron out the edges and squash the easy targets.

I’m not sure if the small things are worth slowing down studies for, but something medium sized might be. So, I’ve been toying around with the idea of building a some sort multiplayer workspace thing where you have a proper in-game code editor and tools for helping quick (should I say, rabid?) prototyping and playing around with physics objects.

JanusVR has been doing great work on something very much like this plus it’s a sort of a web browser. The problem is, it’s closed source. I understand it could be because the creator wants to avoid the overhead of managing an open source project and wants to finish the thing first. But I can’t just sit and wait while he does that and hope it comes out alright, whenever that will be.

So maybe I should just do it myself, with the focus I’d like it to have. JanusVR’s in-game editor looked really ugly and unusable anyway, so with a different focus for my own project, it could actually bring something new to the table rather early in the process. And obviously my efforts would be open source from the start.

One additional idea I’ve been thinking about including in the project, is maybe combining the whole thing with something resembling a REST API to help communicating between the inside and the outside of the game. Think tweets causing new objects to spawn etc., and events in the “game” causing tweets.