In a notable shift within the AI industry, OpenAI has reportedly begun renting Google’s Tensor Processing Units (TPUs) to power ChatGPT and other AI-driven services.
Highlights
- Infrastructure Diversification: OpenAI is reportedly renting Google Cloud TPUs (Tensor Processing Units) to support ChatGPT and other AI services, marking a major shift away from its historical reliance on Nvidia GPUs and Microsoft Azure.
- Cost and Capacity Drivers: The move is driven by growing compute demands and the rising cost and limited availability of Nvidia GPUs. Google’s TPUs offer better throughput-per-dollar for inference workloads, according to MLPerf benchmarks.
- Using Older TPU Models: OpenAI is reportedly using older-generation Google TPUs (not Google’s latest TPU models), which may deliver lower performance compared to Nvidia’s H100 GPUs but still help ease capacity constraints.
- Reduced Azure Dependence: The shift signals a strategic move away from exclusive dependence on Microsoft Azure for cloud infrastructure needs.
- Google Cloud’s Growing TPU Client List: OpenAI joins other high-profile Google Cloud TPU customers like Apple, Anthropic, and Safe Superintelligence—highlighting Google’s broader push to commercialize its AI hardware services.
- Industry Trend: Competitors Becoming Customers: OpenAI’s decision reflects a larger AI industry pattern where even direct competitors collaborate for infrastructure, driven by scale, cost, and availability challenges.
- No Official Comments: As of now, neither OpenAI nor Google has publicly confirmed the reported partnership.
The move reflects OpenAI’s growing need for scalable compute resources amid ongoing shortages and rising costs of Nvidia GPUs.
Moving Beyond Nvidia and Microsoft Azure
For years, OpenAI has relied heavily on Nvidia GPUs, using them for both training large-scale AI models and running inference—the process where models generate outputs in response to new queries.
Much of this infrastructure has traditionally been hosted on Microsoft Azure, OpenAI’s primary cloud partner and investor.
However, with demand for compute resources surging and Nvidia’s latest chips both expensive and in limited supply, OpenAI appears to be diversifying its infrastructure strategy.
According to reports from Reuters and The Information, OpenAI has now started renting Google Cloud TPUs—marking the company’s first large-scale use of non-Nvidia hardware for inference workloads.
This shift also reduces OpenAI’s exclusive reliance on Microsoft Azure, spreading its operations across multiple cloud providers.
Why Google?
OpenAI’s move to Google’s TPUs appears driven by two main factors: cost-efficiency and availability.
- Cost Considerations: TPUs are generally more affordable for inference workloads. Industry benchmarks, including MLPerf, show that Google’s Cloud TPU v5e offers up to 2.7× better throughput-per-dollar compared to its previous TPU v4 generation.
- Capacity Needs: As ChatGPT’s user base grows, OpenAI’s need for more scalable and readily available compute power continues to increase.
Despite this new partnership, reports indicate that Google is not offering OpenAI access to its latest TPU models, limiting OpenAI to older-generation TPUs.
This decision may impact performance relative to Nvidia’s state-of-the-art H100 GPUs, but it still provides OpenAI with much-needed additional capacity.
Google Cloud’s Expanding TPU Client Base
This development also signals Google Cloud’s broader ambition to commercialize its TPU infrastructure. Historically reserved for Google’s internal teams, TPUs have only recently become available to external customers.
OpenAI now joins a growing list of high-profile TPU clients, including Apple, Anthropic, and Safe Superintelligence—the latter two led by former OpenAI executives.
For Google Cloud, onboarding such prominent AI players—even direct competitors—aligns with its goal of expanding revenue from cloud infrastructure services.
Competitors Becoming Customers
OpenAI’s decision to rent compute power from Google highlights an evolving dynamic in the AI sector:
- Competitors Collaborating for Infrastructure Needs: Even though OpenAI and Google are rivals in generative AI products, infrastructure limitations and cost considerations are prompting unexpected partnerships.
- Vendor Diversification: OpenAI’s move also reflects a larger industry trend of avoiding single-provider dependencies, as AI workloads continue to scale in size and cost.
Neither OpenAI nor Google has publicly commented on the reported arrangement.
