Music processing in the Cloud

[Theo Verelst]

Recently I signed up for the free tier of Amazon Web Services, which is by chance, I suppose there are others, too. It's fun to have a "little" Linux box available somewhere stashed away in some big data center that executes your commands. In fact I put on a purely for testing web server, which worked all right, and because I felt up to it I decided to compile the Linux audio tools "jackd" an "jack-rack" on the single small processing node's file system. It fit. And it compiled o.k., so this night I made my first preparatory efforts to do audio processing in the cloud, using some of the same tools my (very) big studio processing graphs apply to start to make digital more bearable!

 

I recall there was a Roland service on the web to improve instrument quality on midi-d tracks, and I saw one company in Minnetonka offering some relatively simple audio processing where you can use their software for a fee, and pay for the (limited) cloud usage as well.

 

Anyone here use a cloud audio service, or have interesting suggestions what you might find interesting?

 

There's the big advantage of the scaling and the infra structure of a big server farm you home PC or Mac is soon going to not win of, like there's big (mostly Intel Xeon) processors with huge memory (even up to a TB) strong and well maintained storage with potentially huge bandwidth and especially, connectivity with many other potent compute nodes over high bandwidth networks. My primary interest however, directly synthesizer related, is the existence of big FPGA boards connected with potent host computer and 30Gb/s network links and fast storage, which can be used per the hour (order of magnitude dollars per hour), as well as freely usable programming tools for the FPGA. Free in terms of no licence fee, the (not so small) computers that run for instance the C-to_FPGA tools also cost per hour, of course.

 

Anyhow, just like big data applications with graphics cards, these huge and very fast FPGAs also lend them to do audio processing with.

 

Of course I've been thinking about getting my studio processing tools in process to run on the big cloud machines, so that I can run the three phases I require to make digital A grade music sound good in parallel, and potentially as a service. That's not happened, no unclearness here, but the main tools appear to potentially run, as well as the facilities allow it, is my conclusion.

 

 

T.

[mate stubb]

The last software company I worked for before I retired wrote a very complex custom app, portions of which run on both Google App Engine and Amazon Web Services. App Engine is more specifically tuned to scale up instantly and is used by the main web app.

 

As a developer, you have to control statefulness and the chunkiness of your process to run efficiently on Google. The reward is huge scalability that is very dynamic. Amazon (at least at the time I was involved, 3 years ago now) is a more traditional web server. The advantage there is that you can run any server app without having to learn new techniques. It's more suitable for batch jobs.

 

 

[TommyRude]

Slightly off topic. If AWS goes down, the entire world as we know it may collapse. Pretty much everything we do and see nowadays is running on AWS, business and financial transactions, services of all kinds, ordering, delivering, advertising, transportation, energy, most of the websites we know and love, I think even the military, cray cray cray.

 

Good luck with the project.

[Theo Verelst]

Just a little progress update for those with the interest, so I took the free tier Linux AWS AMI, which is like a small but decent unix running on a single Zeon 2.4 GHz thread (!) with a small amount of free drive space, and tried to compile/get my very much used audio tools on it. The standard Linux running on it doesn't have repositories for the main tools like jack, jack-rack, etc., so I decided to try to compile them, which has worked, and I also tried to compile "meterbridge" (these are all standard Linux audio tools you should find easily with a search engine), which took a little wrestling and another package compile, but I now could remotely on the cloud computer start a virtual sound server "jackd", open a graphical interface rack with a choice of dozens of plugins on my own machine (running over a secure X windows connection) and even add a meterbride, connect them up and see an audio signal (I tried "crackle" on a "old sound" plugin forthe moment). It took about 1.3MB/s to keep those meters updated on my screen, but it worked!

 

So now what's the connection? Well, I'd like to try to run some of my complicated studio processing on the cloud, and it can be handy to define a service, for instance as I recall Roland has to render a Midi file from a keyboard, and run ti through a much more high quality sampler/processor/etc on the cloud.

 

T.