The AI GPU wars intensify as demand surges

GPUs like those from Nvidia have become essential for training modern AI systems. But progress in artificial intelligence is bumping up against the physical constraints of today's computing systems.

In recent years, graphical processing units, or GPUs, have become integral to powering artificial intelligence systems. Originally created for gaming and graphics applications, GPUs were found about a decade ago to be very effective at training AI models, thanks to their ability to perform parallel processing.

Ronen Dar, co-founder and CTO at Run:ai, said in a recent interview that GPUs have rapidly advanced to become customized for AI workloads. While Nvidia currently dominates the AI chip market with an estimated 87% market share, competitors like AMD and startups are also developing GPUs tailored specifically for AI. Nvidia occasionally releases new GPU hardware generations that push the envelope regarding capabilities for training state-of-the-art AI models. For example, while only two GPUs were required to train top image recognition models ten years ago, rumors suggest OpenAI used over 20,000 GPUs to train the latest GPT-4 model.

“GPUs are essentially the accelerator hardware that can do a lot of parallel processing very efficiently,” said Dar. ‘This makes them really good for ai AI training.”

However, while GPU hardware has rapidly grown more powerful, software frameworks have lagged behind in efficiently utilizing these resources. Oftentimes, utilization rates can be as low as 10-20%, presenting major opportunities for software optimizations.

As AI advances towards more capable systems like artificial general intelligence, more data, computing power, and optimizations will be required. In the next few years, expect to see GPUs become even more integral as startups build novel applications powered by state-of-the-art natural language models and generative AI. The interfaces and experiences we have with computers today will likely be transformed thanks to GPU-enabled AI capabilities just around the corner.