Memo reveals Meta Platforms’ plan to launch in-house custom chips this year for AI advancement.
According to an internal company document viewed by Reuters on Thursday, Meta Platforms, the owner of Facebook, intends to introduce a new iteration of a specialized chip in its data centers this year. The chip is designed to bolster its efforts in artificial intelligence (AI).
The chip, the second generation of a line of in-house silicon that Meta announced last year, could help reduce Meta’s reliance on market-dominant Nvidia chips and manage rising costs associated with AI workloads as it competes to launch AI products.
The world’s largest social media company has sought to add computing power to the powerful generative artificial intelligence products it pushes into apps like Facebook, Instagram and WhatsApp, as well as hardware such as its Ray-Ban smart glasses, spending billions of dollars to amass arsenals of specialized chips and reconfigure data centers for them.
We are on WhatsApp channels. Click to join.
According to Dylan Patel, founder of the silicon research group SemiAnalysis, at the scale at which Meta operates, the successful deployment of its own chip could potentially reduce hundreds of millions of dollars in annual energy costs and billions in chip purchasing costs.
The chips, infrastructure, and energy needed to run AI applications have become a giant investment sink for tech companies, somewhat offsetting the gains made in the rush of excitement around the technology.
A Meta spokesperson confirmed the plan to put the updated chip into production in 2024 and said it would work with hundreds of thousands of off-the-shelf graphics processing units (GPUs) — AI chips. was buying.
“We see our internally developed accelerators as a good complement to commercially available GPUs, providing an optimal combination of performance and efficiency for Meta-specific workloads,” the spokesperson said in a statement.
Meta CEO Mark Zuckerberg said last month that the company plans to receive about 350,000 flagship “H100” processors by the end of the year from Nvidia, which makes the most sought-after GPUs used in artificial intelligence. Together with other suppliers, Meta would accumulate a total of 600,000 H100-equivalent computing capacity, he said.
The introduction of its own chip as part of this plan is a positive turn for Meta’s internal AI silicon project after executives decided in 2022 to spin off the first iteration of the chip.
Instead, the company decided to buy billions of dollars worth of Nvidia GPUs, which has a near-monopoly on an AI process called training, which involves feeding models huge sets of data to teach them how to perform tasks.
The new chip, internally called “Artemis,” like its predecessor, can only perform a process known as inference, in which models are asked to use their algorithms to estimate placement and generate responses to user prompts.
Reuters reported last year that Meta is also working on a more ambitious chip that, like GPUs, would be able to perform both training and inference.
The Menlo Park, Calif.-based company shared data on the first generation of its Meta Training and Inference Accelerator (MTIA) program last year. The announcement presented this version of the chip as a learning opportunity.
Despite those early stumbles, the inference chip could be significantly more efficient at crushing Meta’s featured models than power-hungry Nvidia processors, according to Patel.
“There’s a lot of money and power being used that could be saved,” he said.
