Google’s cheaper, faster TPUs are here, while users of other AI processors face a supply crunch

Relief could be on the way for enterprises facing shortages of GPUs to run their AI workloads, or unable to afford the electricity to power them: Google will add Ironwood, a faster, more energy-efficient version of its Tensor Processing Unit (TPU), to its cloud computing offering in the coming weeks.

Analysts expect Ironwood to offer price-performance similar to GPUs from AMD and Nvidia, running in Google’s cloud, so this could ease the pressure on enterprises and vendors struggling to secure GPUs for AI model training or inferencing projects.

That would be particularly welcome as enterprises grapple with a global shortage of high-end GPUs that is driving up costs and slowing AI deployment timelines, and even those who have the GPUs can’t always get the electricity to operate them.

That doesn’t mean it will be all plain sailing for Google and its TPU customers, though: Myron Xie, a research analyst at SemiAnalysis, warned that Google might also face constraints in terms of chip manufacturing capacity at Taiwan Semiconductor Manufacturing Company (TSMC), which is facing bottlenecks around limited capacity for advanced chip packaging.

Designed for TensorFlow

Ironwood is the seventh generation of Google’s TPU platform, and was designed alongside TensorFlow, Google’s open-source machine learning framework.

That gives the chips an edge over GPUs in general for common in AI workloads built for TensorFlow, said Omdia principal analyst Alexander Harrowell. Many AI models, especially in research and enterprise scenarios, are built using TensorFlow, he said, and the TPUs are highly optimized for such operations while general-purpose GPUs that support multiple frameworks aren’t as specialized.

Opportunities for the AI industry

LLM vendors such as OpenAI and Anthropic, which still have relatively young code bases and are continuously evolving them, also have much to gain from the arrival of Ironwood for training their models, said Forrester vice president and principal analyst Charlie Dai.

In fact, Anthropic has already agreed to procure 1 million TPUs for training and its models and using them for inferencing. Other, smaller vendors using Google’s TPUs for training models include Lightricks and Essential AI.

Google has seen a steady increase in demand for its TPUs (which it also uses to run interna services), and is expected to buy $9.8 billion worth of TPUs from Broadcom this year, compared to $6.2 billion and $2.04 billion in 2024 and 2023 respectively, according to Harrowell.

“This makes them the second-biggest AI chip program for cloud and enterprise data centers, just tailing Nvidia, with approximately 5% of the market. Nvidia owns about 78% of the market,” Harrowell said.

The legacy problem

While some analysts were optimistic about the prospects for TPUs in the enterprise, IDC research director Brandon Hoff said enterprises will most likely to stay away from Ironwood or TPUs in general because of their existing code base written for other platforms.

“For enterprise customers who are writing their own inferencing, they will be tied into Nvidia’s software platform,” Hoff said, referring to CUDA, the software platform that runs on Nvidia GPUs. CUDA was released to the public in 2007, while the first version of TensorFlow has only been around since 2015.

This article first appeared on Network World.