Music Player Network
Previous Thread
Next Thread
Print Thread
GPUs for audio tasks
#3039618 04/19/20 06:39 PM
Joined: Jan 2001
Posts: 2,803
Likes: 26
But who is number 1 ? ...
MP Hall of Fame Member
OP Offline
But who is number 1 ? ...
MP Hall of Fame Member
Joined: Jan 2001
Posts: 2,803
Likes: 26
I suspect there’s a good answer to this from the more engineering-savvy folks, but I’ve always wondered why more DAWs and plug-ins don’t offer the option to grab processing power from a computer’s installed GPU. The original UAD-1 accelerator was based on a graphics chip, after all. I imagine the sharing of tasks between CPU and GPU is complex, but it seems like the demand would be there. Imagine being able to expand the number of plug-ins you can run by buying a couple of used 1080s on eBay, for example. Is there a reason we don't hear at least more conversation about this?

Last edited by Stephen Fortner; 04/19/20 06:40 PM.

"I'm just a confused musician who got sidetracked into this damned word business..." -Hunter S. Thompson

Stephen Fortner
Principal, Fortner Media
Senior Editor, Music Player Network
Former Editor in Chief, Keyboard Magazine

Re: GPUs for audio tasks
Stephen Fortner #3040026 04/21/20 11:00 PM
Joined: Jan 2015
Posts: 458
Likes: 11
N
Senior Member
Offline
Senior Member
N
Joined: Jan 2015
Posts: 458
Likes: 11
Yes. Audio is real-time and processing must be sequential. This means that if I put an EQ before the compressor, then the audio bit stream has to pass through the EQ before going into the compressor code. GPU's are not designed for this, despite being massively powerful.

GPUs are designed for massively parallel workloads. Think about a typical game - I have to paint an entire 4k screen, using thousands of texture files, lighting FX, etc. But every pixel can be calculated independently of any other pixel. So, if I can throw a couple thousand processor cores at it, so much the better - I can finish sooner. GPUs are optimized for a condition exactly opposite of audio.

Audio doesn't parallelize this way. Your DAW's master fader is running on a single core, no matter how many you have. It has to.

If you want to play with this, put lots of complicated bussing and routing in a project. Watch the real-time CPU performance meter go up, even when you aren't doing anything. Then take all the complex routing out and bypass your plugins, and watch the real-time CPU get real skinny. If you want to have a super low latency tracking setup, disable all plugins, eliminate the bussing, and I bet you can run a smaller buffer. Then load up your full mix template. You may have to increase your buffer to keep from getting dropouts. This is the real-time requirement for audio at work inside a non-realtime OS (Windows, OS X, Linux).

In audio projects, even big orchestral mockups rarely drive more than 50% absolute CPU usage. But it is quite possible to run out of real-time assets on a CPU that is only running at 50% actual utilization. Audio has to work at a constant rate (the sample rate). Almost nothing else in the computer works that way. So when the computer has to switch back and forth rapidly from thing to thing, for our purposes, it can only work for audio as long as it can keep the audio buffers full. This is always much less than 100% of CPU power.

GPUs work great for graphics. But it isn't industry oversight that they aren't used for audio. If anyone could use them, UAD would have been put under years ago, or stopped making DSP hardware.

DSP and FPGA's work for audio because they are constant latency devices. When the audio path is burned into silicon, then the whole chip runs real-time. This is why Avid and others have been so committed to the idea. DSP is GREAT for audio. DSP is not nearly as powerful as a modern CPU, but it doesn't get distracted all the time. That's why live sound mixing desks all use FPGA's. They are programmable and re-configurable, but they are fixed latency, once loaded.

All the analogies between audio and video (or photography) really just aren't very true. Audio is quite different.

Re: GPUs for audio tasks
Stephen Fortner #3040027 04/21/20 11:15 PM
Joined: Jan 2015
Posts: 458
Likes: 11
N
Senior Member
Offline
Senior Member
N
Joined: Jan 2015
Posts: 458
Likes: 11
Audio is actually very low bandwidth for modern CPUs. It is more that our modern OS's distract them to death. Merging's DAW hacks Windows to leave several cores completely and totally alone, letting them schedule audio processing far more efficiently.

My studio runs on Dante. Every Dante interface card is really just an FPGA and supporting memory, hardware, etc. The PCIe version runs 128ch of 96khz audio with sub 1ms latency - why? Its a fixed latency, real-time device. Of course, by the time my 8C/16T Intel CPU gets involved, even though it is WAY more powerful, it has to take 3-5 times longer to run those tracks just because it is distracted by thousands of other threads that all want attention.

Sadly music is too small a space to demand a purpose built real-time OS. If we could have a true real-time OS for our DAW computers, everyone would think a modern computer was many times more powerful than we experience. The truth is that we use computers that do audio incidentally. They are far more likely to optimized for graphical performance, hence the long history of audio interfaces with DSP mixers, FPGAs, etc. It doesn't solve the CPU problem, but at least gives an environment where audio is a first class citizen and is treated appropriately. UAD is a master at marketing this, obviously, but so is RME with TotalMix, etc. All the live mixing desks are switching from DSP to FPGA's - that's why there's now all kinds of emulation plugins available for them. Notice there's always a fixed amount of them? Why? Because its hardware - there's only so many gates that be programmed for that use - the others are used. But the latency is constant - it doesn't cost more gates to keep performance. That's very cool and how we want audio to work.

Avid HDX is the right architectural approach for audio - they just botched the license model in the transition from HD, and then computers got "mostly good enough for most things" to the point where few actually NEED fixed, essentially zero latency audio processing. This is still a big deal if you need to push record on 80 armed tracks at an orchestral session. But for anyone working at home one instrument at a time? Meaningless. And so, HDX has largely been sidelined, and Avid has pretty much equal performance on the Native side for most people and most uses. UAD offered branded emulations on hardware that worked with any DAW and took Avid's market outside of film and professional audio (which is a very small scene, actually). The whole world of amateur, hobby, prosumer, gigging, self-producing people (which includes some very big, very profitable acts) works just fine on a UAD accelerator.

Re: GPUs for audio tasks
Stephen Fortner #3040028 04/21/20 11:28 PM
Joined: Jan 2015
Posts: 458
Likes: 11
N
Senior Member
Offline
Senior Member
N
Joined: Jan 2015
Posts: 458
Likes: 11
In a world of "Why of course I like riding my flying unicorn!", we would all want there to be an industry standard real-time OS dedicated to music. Every DAW maker would run on this one OS, and it would work on any Intel platform. If we had this, something like the UAD accelerators would all go away almost instantly, and UAD would port their code to this OS as plugins. We would be drowning in CPU.

Software synth manufacturers could radically improve their code given all the extra cycles freed up. No one would worry about audio interface latency for tracking. Drivers would have minimal buffering.

But you are an industry insider. What do you think are the odds that all the DAW and plugin manufacturers would agree on spending a couple years to make this OS? Or that one would and then freely share it with others? And they would take it up without modification so that no one would struggle with compability?

Windows, and OS X are ubiquitous, "free" from a developer standpoint, and good enough for 99% of all use cases. Audio processing beyond the realm of human sensory perception has been real for well over a decade, probably two, so it is just isn't a "problem" that is likely to get any attention. But if it did, all musicians who work with digital audio would rejoice.

Re: GPUs for audio tasks
Nathanael_I #3040074 04/22/20 11:28 AM
Joined: Oct 2017
Posts: 385
Likes: 4
Senior Member
Offline
Senior Member
Joined: Oct 2017
Posts: 385
Likes: 4
Originally Posted by Nathanael_I
Yes. Audio is real-time and processing must be sequential. This means that if I put an EQ before the compressor, then the audio bit stream has to pass through the EQ before going into the compressor code. GPU's are not designed for this, despite being massively powerful.

GPUs are designed for massively parallel workloads. Think about a typical game - I have to paint an entire 4k screen, using thousands of texture files, lighting FX, etc. But every pixel can be calculated independently of any other pixel. So, if I can throw a couple thousand processor cores at it, so much the better - I can finish sooner. GPUs are optimized for a condition exactly opposite of audio.

Audio doesn't parallelize this way. Your DAW's master fader is running on a single core, no matter how many you have. It has to.

It doesn't parallelise this way? Logic runs each channel strip in a separate thread. So my project with the 127 tracks and sixteen busses is actually a massively parallel operation of 144 threads.

Or do GPUs offer WAY more cores than that, so the parallelity is just overkill?

Re: GPUs for audio tasks
Nathanael_I #3040078 04/22/20 11:58 AM
Joined: Feb 2005
Posts: 420
Likes: 7
Senior Member
Offline
Senior Member
Joined: Feb 2005
Posts: 420
Likes: 7
Originally Posted by Nathanael_I
In a world of "Why of course I like riding my flying unicorn!", we would all want there to be an industry standard real-time OS dedicated to music. Every DAW maker would run on this one OS, and it would work on any Intel platform. If we had this, something like the UAD accelerators would all go away almost instantly, and UAD would port their code to this OS as plugins. We would be drowning in CPU.

Around 2000, there was an operating system developed that was expressively tailored for music systems. I don't remember its name now (and can't Google it because I don't remember its name) but I'll bet The Real Dr. Mike remembers it. TASCAM developed at least one product using that OS, the SX-1 workstation. They had a significant investment in the software developer, but apparently it fizzled out rather quickly.

Nearly all users want audio hardware that runs on the computer that they use for everything else.

Re: GPUs for audio tasks
analogika #3040102 04/22/20 02:44 PM
Joined: Jan 2015
Posts: 458
Likes: 11
N
Senior Member
Offline
Senior Member
N
Joined: Jan 2015
Posts: 458
Likes: 11
Originally Posted by analogika
It doesn't parallelise this way? Logic runs each channel strip in a separate thread. So my project with the 127 tracks and sixteen busses is actually a massively parallel operation of 144 threads.

Or do GPUs offer WAY more cores than that, so the parallelity is just overkill?

Actually, your example illustrates the point. Your DAW actually uses many more threads than 144, but this doesn't make it a massively parallel, GPU-ready workload. In your example you have 127 tracks, 16 busses, and presumably a master fader (stereo, 5.1, whatever). But it isn't 144 discrete workloads. Each of the 127 tracks can be computed in a single, un-correlated thread, as you suggest. But what of the buses? A bus thread can't process data until it gets valid samples from every single track feeding it. The master fader can't run until every bus feeding it has given it processed samples. The whole "massively parallel" thing breaks down very quickly into a small number of threads, any one of which can be the "long pole in the tent" that everything else waits for. The whole DAW can only go as fast as it can get every thread serviced in the exact right order and still keep the sound-card buffers full.

A single "slow" thread can cause the whole thing to gap. This is why you can get "cracks and pops" while your CPU is running 48% utilization in absolute processing terms. It just can't switch between all the threads any faster, even though it could handle more data on every clock cycle. Audio is real-time. I can't just give it 5 min of data to do at once. I have to give it one slice at a time, at a constant data rate.

I don't know if you use Mainstage, but you used to see the limits of "one thread per track" quite clearly. A big plugin could end up crackling just due to needing more than one core could offer in real-time performance.

Now, in practice? Your modern CPU will laugh off the 144 threads, 16 busses and master fader on a 4-8 core machine unless you do some massive processing or have the busses nested into each other in complex ways. Your machine runs thousands of threads many, many times every second. Modern CPUs are astonishingly powerful compared to audio bandwidth.

I think this is Mike River's point (and a larger point on adequacy). Most people don't want a dedicated music computer, even if it would be "better for audio". They do want the one device they paid for to handle audio too. And it does for most people and most uses. For some, adding a UAD card solves any other issues, and for the very few, Avid's HDX handles the rest.

The truth is that DAW's can't even use all the cores of a modern Threadripper chip. Most DAW's top out at 24-32 cores. You can see this in the latest benchmarks over at ScanPro.

The other thing that is problematic for GPU processing of audio is the memory latency. GPU's work for games because many of the textures and such (the raw materials) are loaded into the 6-12GB of very fast RAM inside the GPU. Again, audio doesn't work this way. You can't pre-load the audio stream. It has to be computed in real-time. So the CPU would have to constantly feed data to the GPU only to buffer it and get it back later. It is easier to just process it on the CPU since it isn't "that much" data.

So, there is a reason that no one uses the GPU. It isn't well suited for the task, and audio processing is "good enough" on a non-real-time CPU for all but extreme corner cases.

This comes up on the composers forum at vi-control about every other year. But the software engineers have known about GPU for a very long time. If it was useful, it would have been used ages ago. Engineers love efficiency. They are also very pragmatic about "good enough", and we have more than "good enough" in general purpose CPUs.

1 member likes this: analogika
Re: GPUs for audio tasks
Mike Rivers #3040368 04/24/20 04:19 PM
Joined: Jun 2019
Posts: 606
Likes: 29
Gold Member
Offline
Gold Member
Joined: Jun 2019
Posts: 606
Likes: 29
Originally Posted by Mike Rivers
Originally Posted by Nathanael_I
In a world of "Why of course I like riding my flying unicorn!", we would all want there to be an industry standard real-time OS dedicated to music. Every DAW maker would run on this one OS, and it would work on any Intel platform. If we had this, something like the UAD accelerators would all go away almost instantly, and UAD would port their code to this OS as plugins. We would be drowning in CPU.

Around 2000, there was an operating system developed that was expressively tailored for music systems. I don't remember its name now (and can't Google it because I don't remember its name) but I'll bet The Real Dr. Mike remembers it. TASCAM developed at least one product using that OS, the SX-1 workstation. They had a significant investment in the software developer, but apparently it fizzled out rather quickly.

Nearly all users want audio hardware that runs on the computer that they use for everything else.
Are you thinking of BeOS? That poor thing came and went in a tearing hurry... nice concept, NO traction in that market.

What we in the electronic music biz, especially in software, call "poof.gone". (said rapidly in one two-syllable expression)


Dr. Mike Metlay (PhD in nuclear physics, golly gosh) grin
Janitor and Hall Monitor, Dr. Mike's Studio Workshop

clicky!: more about me ~ my schwag ~ my radio station (and my fam) ~ my local tribe ~ my day job
Re: GPUs for audio tasks
Stephen Fortner #3040427 04/24/20 09:09 PM
Joined: Jan 2015
Posts: 458
Likes: 11
N
Senior Member
Offline
Senior Member
N
Joined: Jan 2015
Posts: 458
Likes: 11
Yeah that's it, that's what was in the back of my head writing this. I didn't even bother to put the name in because I don't know that it was ever even released. Audio is so low bandwidth compared to video that it kind of just rides for free now. In a Windows 95 world a real-time OS probably seemed like a very obvious need for things to progress. Moore's Law did the work, however, and quickly made it so that even though nothing was real-time everything was fast enough for most uses that the distinction has long since become unnecessary.

Much of music making is done by essentially non-technical people. Computer literate, for sure, but not "code enabled". One would think that a developer could release software as an ISO image, running some massively stripped out version of Linux where there is little to interrupt the CPU. But most wouldn't dedicate a machine to music, rendering the idea moot.

So, "unicorns and rainbows, and all". Here, I use Win10 and OS X for music making, both quite successfully. So it is another one of those things where there is a technically "better" architecture, but in the real world just doesn't amount to much.

I suspect that in the next two years, the major DAWs will all figure out how to take advantage of the new massive core counts that are happening. Now that we aren't going to get faster clocks, we are going to get more cores. I'm on an I9 9900k (8C/16T) right now. The new Threadripper stuff has so many cores that DAWBench won't even run correctly, let alone something like Cubase! That will all get worked out.

There is a rumor that the optional $2,000 Afterburner video accelerator card for the new Mac Pro is an FPGA platform and that therefore, someone could code audio algorithms into it ala UAD. But how many musicians buy an $8-15k computer and then put another $2k into that card just to avoid buying a UAD card or HDX, both of which have quite a few other benefits?

What I know is that it is a great time to be making music - the tech, the price points, the quality have never been better.

Re: GPUs for audio tasks
Dr Mike Metlay #3040455 04/24/20 11:21 PM
Joined: Feb 2005
Posts: 420
Likes: 7
Senior Member
Offline
Senior Member
Joined: Feb 2005
Posts: 420
Likes: 7
Originally Posted by Dr Mike Metlay
Are you thinking of BeOS? That poor thing came and went in a tearing hurry... nice concept, NO traction in that market.

What we in the electronic music biz, especially in software, call "poof.gone". (said rapidly in one two-syllable expression)

That's it. I knew you'd remember or have a note about it somewhere. It seemed like it had a lot going for it if you were building a dedicated MIDI/Audio product like TASCAM's, but without a chicken, you got no eggs.


Moderated by  Dr Mike Metlay 

Link Copied to Clipboard
Powered by UBB.threads™ PHP Forum Software 7.7.4