Hugging Face integrates native TPU support for PyTorch
In brief
- TorchTPU adds a native backend to run PyTorch on TPUs without intermediate solutions.
- Hugging Face's Transformers and Diffusers now support TPUs with device_map="tpu".
- Performance gains reach up to 21.5x faster for some large-scale models.
Hugging Face introduces TorchTPU, a native PyTorch backend enabling direct execution of models on TPUs without relying on intermediate solutions like PyTorch/XLA or JAX.
TorchTPU adds TPU as a standard device in PyTorch, featuring optimized kernels (Pallas) and torch.compile integration for performance gains. Hugging Face's Transformers and Diffusers libraries now support TPUs, allowing models to be loaded with device_map="tpu" and run in eager or compiled mode. Large models can be distributed across multiple TPUs via TensorParallelConfig, and ready-to-use kernels are available on the Kernels Hub.
This integration streamlines the deployment of Hugging Face models on TPUs, particularly for inference and large-scale training. Tests show significant performance improvements, such as 21.5x faster decoding for Qwen3-30B-A3B at batch size 64. Hugging Face also publishes a CI dashboard to track TPU test results, currently showing 96.5% success.
This summary gives you the essentials. The full article, its figures and quotes are on Hugging Face - Blog.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error