Jump to content


Please note: You can easily log in to MPN using your Facebook account!

GPUs for audio tasks


Recommended Posts

I suspect there"s a good answer to this from the more engineering-savvy folks, but I"ve always wondered why more DAWs and plug-ins don"t offer the option to grab processing power from a computer"s installed GPU. The original UAD-1 accelerator was based on a graphics chip, after all. I imagine the sharing of tasks between CPU and GPU is complex, but it seems like the demand would be there. Imagine being able to expand the number of plug-ins you can run by buying a couple of used 1080s on eBay, for example. Is there a reason we don't hear at least more conversation about this?

Stephen Fortner

Principal, Fortner Media

Former Editor in Chief, Keyboard Magazine

Digital Piano Consultant, Piano Buyer Magazine

 

Industry affiliations: Antares, Arturia, Giles Communications, MS Media, Polyverse

 

 

Link to comment
Share on other sites



  • Replies 14
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Yes. Audio is real-time and processing must be sequential. This means that if I put an EQ before the compressor, then the audio bit stream has to pass through the EQ before going into the compressor code. GPU's are not designed for this, despite being massively powerful.

 

GPUs are designed for massively parallel workloads. Think about a typical game - I have to paint an entire 4k screen, using thousands of texture files, lighting FX, etc. But every pixel can be calculated independently of any other pixel. So, if I can throw a couple thousand processor cores at it, so much the better - I can finish sooner. GPUs are optimized for a condition exactly opposite of audio.

 

Audio doesn't parallelize this way. Your DAW's master fader is running on a single core, no matter how many you have. It has to.

 

If you want to play with this, put lots of complicated bussing and routing in a project. Watch the real-time CPU performance meter go up, even when you aren't doing anything. Then take all the complex routing out and bypass your plugins, and watch the real-time CPU get real skinny. If you want to have a super low latency tracking setup, disable all plugins, eliminate the bussing, and I bet you can run a smaller buffer. Then load up your full mix template. You may have to increase your buffer to keep from getting dropouts. This is the real-time requirement for audio at work inside a non-realtime OS (Windows, OS X, Linux).

 

In audio projects, even big orchestral mockups rarely drive more than 50% absolute CPU usage. But it is quite possible to run out of real-time assets on a CPU that is only running at 50% actual utilization. Audio has to work at a constant rate (the sample rate). Almost nothing else in the computer works that way. So when the computer has to switch back and forth rapidly from thing to thing, for our purposes, it can only work for audio as long as it can keep the audio buffers full. This is always much less than 100% of CPU power.

 

GPUs work great for graphics. But it isn't industry oversight that they aren't used for audio. If anyone could use them, UAD would have been put under years ago, or stopped making DSP hardware.

 

DSP and FPGA's work for audio because they are constant latency devices. When the audio path is burned into silicon, then the whole chip runs real-time. This is why Avid and others have been so committed to the idea. DSP is GREAT for audio. DSP is not nearly as powerful as a modern CPU, but it doesn't get distracted all the time. That's why live sound mixing desks all use FPGA's. They are programmable and re-configurable, but they are fixed latency, once loaded.

 

All the analogies between audio and video (or photography) really just aren't very true. Audio is quite different.

Link to comment
Share on other sites

Audio is actually very low bandwidth for modern CPUs. It is more that our modern OS's distract them to death. Merging's DAW hacks Windows to leave several cores completely and totally alone, letting them schedule audio processing far more efficiently.

 

My studio runs on Dante. Every Dante interface card is really just an FPGA and supporting memory, hardware, etc. The PCIe version runs 128ch of 96khz audio with sub 1ms latency - why? Its a fixed latency, real-time device. Of course, by the time my 8C/16T Intel CPU gets involved, even though it is WAY more powerful, it has to take 3-5 times longer to run those tracks just because it is distracted by thousands of other threads that all want attention.

 

Sadly music is too small a space to demand a purpose built real-time OS. If we could have a true real-time OS for our DAW computers, everyone would think a modern computer was many times more powerful than we experience. The truth is that we use computers that do audio incidentally. They are far more likely to optimized for graphical performance, hence the long history of audio interfaces with DSP mixers, FPGAs, etc. It doesn't solve the CPU problem, but at least gives an environment where audio is a first class citizen and is treated appropriately. UAD is a master at marketing this, obviously, but so is RME with TotalMix, etc. All the live mixing desks are switching from DSP to FPGA's - that's why there's now all kinds of emulation plugins available for them. Notice there's always a fixed amount of them? Why? Because its hardware - there's only so many gates that be programmed for that use - the others are used. But the latency is constant - it doesn't cost more gates to keep performance. That's very cool and how we want audio to work.

 

Avid HDX is the right architectural approach for audio - they just botched the license model in the transition from HD, and then computers got "mostly good enough for most things" to the point where few actually NEED fixed, essentially zero latency audio processing. This is still a big deal if you need to push record on 80 armed tracks at an orchestral session. But for anyone working at home one instrument at a time? Meaningless. And so, HDX has largely been sidelined, and Avid has pretty much equal performance on the Native side for most people and most uses. UAD offered branded emulations on hardware that worked with any DAW and took Avid's market outside of film and professional audio (which is a very small scene, actually). The whole world of amateur, hobby, prosumer, gigging, self-producing people (which includes some very big, very profitable acts) works just fine on a UAD accelerator.

Link to comment
Share on other sites

In a world of "Why of course I like riding my flying unicorn!", we would all want there to be an industry standard real-time OS dedicated to music. Every DAW maker would run on this one OS, and it would work on any Intel platform. If we had this, something like the UAD accelerators would all go away almost instantly, and UAD would port their code to this OS as plugins. We would be drowning in CPU.

 

Software synth manufacturers could radically improve their code given all the extra cycles freed up. No one would worry about audio interface latency for tracking. Drivers would have minimal buffering.

 

But you are an industry insider. What do you think are the odds that all the DAW and plugin manufacturers would agree on spending a couple years to make this OS? Or that one would and then freely share it with others? And they would take it up without modification so that no one would struggle with compability?

 

Windows, and OS X are ubiquitous, "free" from a developer standpoint, and good enough for 99% of all use cases. Audio processing beyond the realm of human sensory perception has been real for well over a decade, probably two, so it is just isn't a "problem" that is likely to get any attention. But if it did, all musicians who work with digital audio would rejoice.

Link to comment
Share on other sites

Yes. Audio is real-time and processing must be sequential. This means that if I put an EQ before the compressor, then the audio bit stream has to pass through the EQ before going into the compressor code. GPU's are not designed for this, despite being massively powerful.

 

GPUs are designed for massively parallel workloads. Think about a typical game - I have to paint an entire 4k screen, using thousands of texture files, lighting FX, etc. But every pixel can be calculated independently of any other pixel. So, if I can throw a couple thousand processor cores at it, so much the better - I can finish sooner. GPUs are optimized for a condition exactly opposite of audio.

 

Audio doesn't parallelize this way. Your DAW's master fader is running on a single core, no matter how many you have. It has to.

 

It doesn't parallelise this way? Logic runs each channel strip in a separate thread. So my project with the 127 tracks and sixteen busses is actually a massively parallel operation of 144 threads.

 

Or do GPUs offer WAY more cores than that, so the parallelity is just overkill?

"The Angels of Libra are in the European vanguard of the [retro soul] movement" (Bill Buckley, Soul and Jazz and Funk)

The Drawbars | off jazz organ trio

Link to comment
Share on other sites

In a world of "Why of course I like riding my flying unicorn!", we would all want there to be an industry standard real-time OS dedicated to music. Every DAW maker would run on this one OS, and it would work on any Intel platform. If we had this, something like the UAD accelerators would all go away almost instantly, and UAD would port their code to this OS as plugins. We would be drowning in CPU.

 

Around 2000, there was an operating system developed that was expressively tailored for music systems. I don't remember its name now (and can't Google it because I don't remember its name) but I'll bet The Real Dr. Mike remembers it. TASCAM developed at least one product using that OS, the SX-1 workstation. They had a significant investment in the software developer, but apparently it fizzled out rather quickly.

 

Nearly all users want audio hardware that runs on the computer that they use for everything else.

Link to comment
Share on other sites

 

It doesn't parallelise this way? Logic runs each channel strip in a separate thread. So my project with the 127 tracks and sixteen busses is actually a massively parallel operation of 144 threads.

 

Or do GPUs offer WAY more cores than that, so the parallelity is just overkill?

 

Actually, your example illustrates the point. Your DAW actually uses many more threads than 144, but this doesn't make it a massively parallel, GPU-ready workload. In your example you have 127 tracks, 16 busses, and presumably a master fader (stereo, 5.1, whatever). But it isn't 144 discrete workloads. Each of the 127 tracks can be computed in a single, un-correlated thread, as you suggest. But what of the buses? A bus thread can't process data until it gets valid samples from every single track feeding it. The master fader can't run until every bus feeding it has given it processed samples. The whole "massively parallel" thing breaks down very quickly into a small number of threads, any one of which can be the "long pole in the tent" that everything else waits for. The whole DAW can only go as fast as it can get every thread serviced in the exact right order and still keep the sound-card buffers full.

 

A single "slow" thread can cause the whole thing to gap. This is why you can get "cracks and pops" while your CPU is running 48% utilization in absolute processing terms. It just can't switch between all the threads any faster, even though it could handle more data on every clock cycle. Audio is real-time. I can't just give it 5 min of data to do at once. I have to give it one slice at a time, at a constant data rate.

 

I don't know if you use Mainstage, but you used to see the limits of "one thread per track" quite clearly. A big plugin could end up crackling just due to needing more than one core could offer in real-time performance.

 

Now, in practice? Your modern CPU will laugh off the 144 threads, 16 busses and master fader on a 4-8 core machine unless you do some massive processing or have the busses nested into each other in complex ways. Your machine runs thousands of threads many, many times every second. Modern CPUs are astonishingly powerful compared to audio bandwidth.

 

I think this is Mike River's point (and a larger point on adequacy). Most people don't want a dedicated music computer, even if it would be "better for audio". They do want the one device they paid for to handle audio too. And it does for most people and most uses. For some, adding a UAD card solves any other issues, and for the very few, Avid's HDX handles the rest.

 

The truth is that DAW's can't even use all the cores of a modern Threadripper chip. Most DAW's top out at 24-32 cores. You can see this in the latest benchmarks over at ScanPro.

 

The other thing that is problematic for GPU processing of audio is the memory latency. GPU's work for games because many of the textures and such (the raw materials) are loaded into the 6-12GB of very fast RAM inside the GPU. Again, audio doesn't work this way. You can't pre-load the audio stream. It has to be computed in real-time. So the CPU would have to constantly feed data to the GPU only to buffer it and get it back later. It is easier to just process it on the CPU since it isn't "that much" data.

 

So, there is a reason that no one uses the GPU. It isn't well suited for the task, and audio processing is "good enough" on a non-real-time CPU for all but extreme corner cases.

 

This comes up on the composers forum at vi-control about every other year. But the software engineers have known about GPU for a very long time. If it was useful, it would have been used ages ago. Engineers love efficiency. They are also very pragmatic about "good enough", and we have more than "good enough" in general purpose CPUs.

Link to comment
Share on other sites

In a world of "Why of course I like riding my flying unicorn!", we would all want there to be an industry standard real-time OS dedicated to music. Every DAW maker would run on this one OS, and it would work on any Intel platform. If we had this, something like the UAD accelerators would all go away almost instantly, and UAD would port their code to this OS as plugins. We would be drowning in CPU.

 

Around 2000, there was an operating system developed that was expressively tailored for music systems. I don't remember its name now (and can't Google it because I don't remember its name) but I'll bet The Real Dr. Mike remembers it. TASCAM developed at least one product using that OS, the SX-1 workstation. They had a significant investment in the software developer, but apparently it fizzled out rather quickly.

 

Nearly all users want audio hardware that runs on the computer that they use for everything else.

Are you thinking of BeOS? That poor thing came and went in a tearing hurry... nice concept, NO traction in that market.

 

What we in the electronic music biz, especially in software, call "poof.gone". (said rapidly in one two-syllable expression)

Dr. Mike Metlay (PhD in nuclear physics, golly gosh) :D

Musician, Author, Editor, Educator, Impresario, Online Radio Guy, Cut-Rate Polymath, and Kindly Pedant

Editor-in-Chief, Bjooks ~ Author of SYNTH GEMS 1

 

clicky!:  more about me ~ my radio station (and my fam) ~ my local tribe ~ my day job ~ my bookmy music

Link to comment
Share on other sites

Yeah that's it, that's what was in the back of my head writing this. I didn't even bother to put the name in because I don't know that it was ever even released. Audio is so low bandwidth compared to video that it kind of just rides for free now. In a Windows 95 world a real-time OS probably seemed like a very obvious need for things to progress. Moore's Law did the work, however, and quickly made it so that even though nothing was real-time everything was fast enough for most uses that the distinction has long since become unnecessary.

 

Much of music making is done by essentially non-technical people. Computer literate, for sure, but not "code enabled". One would think that a developer could release software as an ISO image, running some massively stripped out version of Linux where there is little to interrupt the CPU. But most wouldn't dedicate a machine to music, rendering the idea moot.

 

So, "unicorns and rainbows, and all". Here, I use Win10 and OS X for music making, both quite successfully. So it is another one of those things where there is a technically "better" architecture, but in the real world just doesn't amount to much.

 

I suspect that in the next two years, the major DAWs will all figure out how to take advantage of the new massive core counts that are happening. Now that we aren't going to get faster clocks, we are going to get more cores. I'm on an I9 9900k (8C/16T) right now. The new Threadripper stuff has so many cores that DAWBench won't even run correctly, let alone something like Cubase! That will all get worked out.

 

There is a rumor that the optional $2,000 Afterburner video accelerator card for the new Mac Pro is an FPGA platform and that therefore, someone could code audio algorithms into it ala UAD. But how many musicians buy an $8-15k computer and then put another $2k into that card just to avoid buying a UAD card or HDX, both of which have quite a few other benefits?

 

What I know is that it is a great time to be making music - the tech, the price points, the quality have never been better.

Link to comment
Share on other sites

Are you thinking of BeOS? That poor thing came and went in a tearing hurry... nice concept, NO traction in that market.

 

What we in the electronic music biz, especially in software, call "poof.gone". (said rapidly in one two-syllable expression)

 

That's it. I knew you'd remember or have a note about it somewhere. It seemed like it had a lot going for it if you were building a dedicated MIDI/Audio product like TASCAM's, but without a chicken, you got no eggs.

Link to comment
Share on other sites

  • 2 months later...

Because we were talking about real-time operating systems and what they would mean for audio - I found out that this is quantifiable. Live mixing consoles typically use either fixed DSP or programmable FPGA's for the audio path. But, SolidState Logic takes a different approach with their Live series consoles and their T-Series broadcast consoles.

 

SSL wrote their own real-time audio operating system to run on commodity PC hardware. The CPU is built into the Live mixing consoles, but in the T-Series, it is in a rack mount server case. The smaller T25 offers 250 fully processed paths; The larger T80 offers 800. Like most high-end digital consoles, the idea of a "path" is just an audio processed path - it could be a channel, a buss, and aux, a return - all processing is available everywhere. You configure whatever combination of inputs, auxes, busses, returns, and outputs that you need out of the pool of available paths. Every path can use any of 8 processing blocks in any order. The truly impressive thing is that the latency is a consistent 8 samples, and it is 100% phase coherent across all channels! The whole console runs as a TDM system. There's no variable processing delay per channel, so everything sums fully phase consistent, with no need for internal delay compensation or large mix buffers. The whole console runs at 64-bit floating point bit depth, so you can feed hundreds of channels into it at full volume and the mix bus will not distort or saturate.

 

If you compare that to something like my A&H SQ-5, it has latency of .7 msec. This is 6,720 67 samples at 96Khz. The SQ-5 is FPGA based, and offers 48 channels and 36 stereo output busses for 84 processed audio paths. This console does not have equal facilities on all track types.

 

From a systems design perspective, SSL has a significant advantage vs. FPGA or DSP-based competitors. SSL doesn't have to deal with silicon development cycles. FPGA and DSP cycles are multi-year design/test/build cycles. Having their own OS, SSL can qualify and release more powerful systems if they want, or replace hardware for the cost of QA cycles. Modern multi-core CPUs are many, many times more powerful than an FPGA or a DSP, and if controlled by a real-time OS - capable of truly amazing results. FPGAs offer more flexibility than DSP, but having your own real-time OS is even better. This is a long term strategic advantage. I'm very impressed with how SSL is running their business. Great products, smart decisions in every part of the markets they serve, great after sales support on the premium stuff - it is a well-run company.

 

Now the SSL consoles are in the $50k and up range with I/O and offer the ability to mix fully immersive audio onto 7.1.4 systems, etc. They are used in top levels of touring, Broadway and West End stages, etc. My little SQ-5 is not competition, though it is quite capable in its own right, delivering latency-free monitor mixes in my studio from about 30 input channels.

 

But if this is possible for a mixing console, you can imagine that DAW's would be radically more stable and performant if a real-time OS did exist for DAWs. I would love a S300-16 console and the T25 Tempest engine in my studio. The DAW would only play back audio onto the Dante output, and the whole mix would be in the SSL. The SSL digital workflows are beautiful and fast. Very clean and intuitive. I would take this over an API or Neve console without hesitation.

Link to comment
Share on other sites

*ahem*

 

The SQ-5 has a latency of 0.7 MILLIseconds, not 0.7 sec.

 

That translates to about 67 samples at 96 kHz, if my math is correct.

 

A .7-second-latency console would be utterly unusable for anything but fully isolated studio use.

"The Angels of Libra are in the European vanguard of the [retro soul] movement" (Bill Buckley, Soul and Jazz and Funk)

The Drawbars | off jazz organ trio

Link to comment
Share on other sites

*ahem*

 

The SQ-5 has a latency of 0.7 MILLIseconds, not 0.7 sec.

 

That translates to about 67 samples at 96 kHz, if my math is correct.

 

A .7-second-latency console would be utterly unusable for anything but fully isolated studio use.

 

Thank you! YES! you are completely correct. I thought it looked funny. I will redline the post. Getting the decimal points right matters!

Link to comment
Share on other sites

Nathanael_I, just wanted to say thanks for all your thorough and detailed information!

Stephen Fortner

Principal, Fortner Media

Former Editor in Chief, Keyboard Magazine

Digital Piano Consultant, Piano Buyer Magazine

 

Industry affiliations: Antares, Arturia, Giles Communications, MS Media, Polyverse

 

 

Link to comment
Share on other sites

Ditto!

Dr. Mike Metlay (PhD in nuclear physics, golly gosh) :D

Musician, Author, Editor, Educator, Impresario, Online Radio Guy, Cut-Rate Polymath, and Kindly Pedant

Editor-in-Chief, Bjooks ~ Author of SYNTH GEMS 1

 

clicky!:  more about me ~ my radio station (and my fam) ~ my local tribe ~ my day job ~ my bookmy music

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...