Summary
Google Cloud has open-sourced TorchTPU, a native PyTorch software stack for training, serving and deploying models on Google TPUs. The release also includes integrations with tools used for model development, inference and distributed training.
Google Cloud announced on October 21, 2026, that TorchTPU, its software stack for running PyTorch on Google TPUs, is open source on GitHub. Co-developed with Meta’s PyTorch team, TorchTPU is intended to let developers train, serve and deploy models on TPUs through familiar PyTorch APIs, with minimal changes to existing workloads.
The release makes TPUs another hardware option within the PyTorch ecosystem, alongside CPUs and GPUs. Google says the aim is to let teams use the same broad set of development and operational tools across different types of computing hardware.
How TorchTPU fits into PyTorch workflows
PyTorch’s eager mode runs operations as they are written, which can make model behaviour easier to inspect while debugging. TorchTPU also supports torch.compile, which can compile and optimise workloads for execution. Google says developers can move from eager-mode debugging to compiled execution, and that the stack includes standard PyTorch APIs, diagnostics and distributed computing primitives.
The announcement describes integrations across model libraries and infrastructure tools. Google and Hugging Face say models from the Hugging Face Hub can run natively on TorchTPU. In Transformers v5, the company says, developers can select a TPU with device_map="tpu". Hugging Face’s Diffusers library is also described as supporting TPU execution, including compiled workloads and tensor parallelism, a method of distributing model computation across multiple accelerators.
For inference, the release includes a native PyTorch backend for vLLM, developed with Inferact. Google says it supports continuous batching, quantization and multi-chip parallelism on TPUs. Continuous batching groups requests efficiently while generation is underway; quantization uses lower-precision representations to reduce model memory or computation requirements.
Google also announced llm-d 1.0, developed with Red Hat, IBM and other contributors. The open-source project coordinates inference across fleets of accelerators. Its router can direct requests to replicas that already hold relevant context, while separating the initial prompt-processing stage from the token-generation stage. On Google Kubernetes Engine, Google says the same llm-d endpoint can be used with GPUs or TPUs.
Training claims and practical significance
For distributed training, TorchTPU is integrated with TorchTitan, a PyTorch-native framework for training models across many accelerators. Google says TorchTitan has been validated on up to 8,000 Ironwood TPUs, also identified as TPU v7x. Separately, the company reports that TorchTPU’s TorchTitan training paths reached up to 52% Model Flops Utilization on Qwen 3 8B, with near-linear scaling and stability across as many as 8,192 Ironwood TPUs. Model Flops Utilization estimates how much of a system’s peak mathematical compute is being used for model operations.
The practical significance is software choice as well as hardware choice. PyTorch teams have often needed framework-specific integration to use a different accelerator family; a native stack and connections to familiar libraries could reduce that friction. Meta said it is deploying PyTorch workloads to Google TPUs for major language-model, advertising and recommendation-system use cases, while Google’s announcement also describes support for serving and training tools used beyond those examples.
The 52% utilization figure is a Google-reported result for a particular model and TPU configuration. The announcement does not provide a comparison baseline, so the figure is best understood as the company’s reported result rather than a direct comparison with other accelerator systems.