Competition for Nvidia’s AI Chips Heats Up Among Startups
Startups are facing a challenge in meeting the high demand for the most essential component of artificial intelligence, causing them to ponder on how they can fulfill AI’s potential.
The lifeblood of generative artificial intelligence is a book-sized semiconductor known as a graphics processing unit (GPU), built by a single company, Nvidia.
Years ago, Nvidia CEO and founder Jensen Huang made a wild bet that the world would soon demand a powerful chip that’s usually used to make video games but could also build artificial intelligence.
No company working on the generative AI models fueling today’s craze can get off the ground without Nvidia’s one-of-a-kind product: the latest model is the H100 and its accompanying software.
This painful reality is one that Amazon, Intel, AMD and others are trying to fix with their own options, but those efforts could take years.
– “Not much GPU” –
And as the biggest tech companies pour all their financial might into generative AI, the smaller fish have to go on the hunt to secure Nvidia’s holy grail.
“Around the world, it’s becoming very difficult to get thousands of GPUs because all these big companies are investing billions of dollars and stockpiling GPUs,” said Fangbo Tao, founder of Singapore-based Mindverse.AI.
“There aren’t a lot of GPUs around,” he said.
Tao was speaking to AFP at the TechCrunch Disrupt conference in San Francisco, where AI startups are flocking to make their pitch to Silicon Valley venture capitalists (VCs).
ChatGPT took the world by storm just as Silicon Valley was suffering from a nasty pandemic-induced hangover, as investors threw money at startups convinced that life had gone irrevocably online.
It turned out to be far-fetched, and the US tech scene went into recession, with layoffs and venture capital funds drying up.
Thanks to AI, some of that old mojo has returned, and anyone with those two letters on their resume is likely to see the red carpet rolled out on legendary Sand Hill Road, home of some of Silicon Valley’s most famous investors.
But when startups walk away with their VC money, the money in their pockets is quickly transferred to Nvidia for GPUs, either directly or through service providers, to realize their AI dreams.
“We call many of the big cloud providers (Microsoft, AWS and Google) and they all tell us that they are having trouble getting supplies,” said Laurent Daudet, CEO of artificial intelligence startup LightOn.
The problem is most acute for companies training generative AI models, which requires power-hungry GPUs to work at peak capacity to process data fed from the Internet.
The computing needs are so huge that very few companies can raise the money to build one of these state-of-the-art large language models.
– “Suck oxygen” –
Microsoft’s $10 billion investment in OpenAI is generally paid in return for purpose-built data centers humming with Nvidia GPUs.
Google has been building its own models, and now Amazon announced Monday that it’s pumping $4 billion into Anthropic AI, another company that trains artificial intelligence.
Working on that mountain of data is “sucking almost all the oxygen out of the GPU market right now,” said Said Ouissal, CEO of Zededa, which aims to reduce AI hunger.
“You’re looking at the middle of next year, maybe the end of next year, before you actually start shipping new orders. The shortage doesn’t seem to be letting up,” added Wes Cummins, CEO of the company’s Applied Digital. which supplies the AI infrastructure.
Companies at the forefront of artificial intelligence also point out that Nvidia’s primary role makes it the de facto kingmaker of the technology’s advancement.
The market is driven “almost entirely by the big players — the Googles, the Amazons, the Metas” with “huge amounts of data and huge amounts of capital,” former Nvidia engineer Jacopo Pantaleoni told The Information.
“This was not the world I wanted to help build,” he said.
Some Silicon Valley veterans said the heady days of Nvidia’s GPUs won’t last forever and that other options will inevitably emerge.
Or the cost of entry will prove too high even for the giants, bringing the current boom to earth.
