Key takeaways
- A TPU is a Google-designed ASIC built to accelerate the matrix math in machine learning, running in Google data centers since 2015.
- Google's docs say TPUs suit large, matrix-heavy models and long training runs, and fit poorly with custom operations or dynamic tensor shapes.
- On November 9, 2026 Google announced TPU 8tx for confidential inference and TPU 8i Max, with twice the compute of TPU 8i, without giving pricing or availability.
What is a TPU? A TPU, or Tensor Processing Unit, is a chip Google designed for one job: the matrix math inside neural networks. Google's Cloud TPU documentation calls it a custom application-specific integrated circuit (ASIC) for accelerating machine learning. Unlike a CPU, it does not try to run every kind of program. Unlike a GPU, it did not start life drawing graphics.
Google now sells TPUs through Google Cloud, and on November 9, 2026 it announced two more: TPU 8tx and TPU 8i Max.
What a TPU does
Neural networks spend most of their time multiplying large grids of numbers, called matrices. A TPU puts that operation into hardware. Its core is a matrix unit (MXU), which Google's docs describe as a 128x128 systolic array. A systolic array is a grid of multiply-and-add cells that pass results to their neighbors, so data flows through the chip without being fetched from memory at every step.
Around the matrix unit sit on-chip high-bandwidth memory (HBM), which lets a TPU hold larger models and batch sizes, and a compiler called XLA. XLA turns the linear algebra, loss and gradient parts of a machine learning program into TPU machine code. The rest of the program runs on the host machine attached to the chip.
TPUs can also be linked into groups the docs call slices, so a workload can scale across many chips with little or no code change.
Where TPUs came from
Google began running TPUs in its data centers in 2015, to speed up neural network inference, meaning running a model that is already trained. The 2017 paper by Norman Jouppi and colleagues describes the first chip. Its matrix unit had 65,536 8-bit multiply-accumulate units, a peak of 92 trillion operations per second and 28 MiB of software-managed on-chip memory.
The authors compared it with an Intel Haswell CPU and an Nvidia K80 GPU running Google's production workloads. They report the TPU was about 15 to 30 times faster and 30 to 80 times better in operations per watt. That is a comparison with 2017 hardware, and Google wrote it about its own workloads. It shows why the chip was built, not how a current TPU compares with a current GPU.
Google's November 9 post says it designed TPUs more than a decade ago for internal work such as voice recognition and translation. They now train and serve Gemini, and Google says TPUs are a growing business with nearly 100 customers, including frontier labs, trading firms, retailers and universities. Google has also started placing TPUs in customers' own data centers.
What is new in TPU 8tx and TPU 8i Max
The eighth generation now has four systems, according to Google's announcement by Amin Vahdat. Google announced TPU 8t and TPU 8i at Google Cloud Next in April. The two added on November 9 are:
- TPU 8tx, for secure and confidential inference. It uses the same silicon as TPU 8t with custom AMD CPU hosts, and builds protection into the hardware so that data and model weights stay isolated while in use. Google says it has up to 2x lower latency than TPU 8t, with a near-zero performance penalty.
- TPU 8i Max, which puts two TPU 8i chips in one package. Google says it has twice the compute, twice the SRAM and twice the HBM bandwidth of TPU 8i, in the same rack footprint. It targets workloads such as agents working across a whole codebase, or hours of video.
| System | Built for | Google's headline claim |
|---|---|---|
| TPU 8t | Large-scale training | Up to 2.7x training performance over the previous generation |
| TPU 8i | Low-latency reasoning | 80% better inference price-performance over the previous generation |
| TPU 8tx | Confidential inference | Up to 2x lower latency than TPU 8t |
| TPU 8i Max | Huge context, dense racks | 2x compute, SRAM and HBM bandwidth over TPU 8i |
All of these figures are Google's own. The post gives no absolute specs such as FLOPS, memory size or power draw, and no pricing or general availability date for TPU 8tx or TPU 8i Max.
TPU vs GPU: when each fits
Google's docs say TPUs suit models dominated by matrix computation, long training runs of weeks or months, large models with large batch sizes, and models with very large embeddings, such as ranking and recommendation systems.
They are a poor fit for programs with frequent branching or many element-wise operations, work that needs high-precision arithmetic, neural networks with custom operations in the main training loop, and models with dynamic tensor shapes, because recompiling when shapes change is slow. For models with custom PyTorch or JAX operations that must partly run on the CPU, the docs point to GPUs.
Software has been the usual barrier. Google says its TorchTPU project, open-sourced in October, runs native PyTorch and Hugging Face models on TPUs without code rewrites.
How to use a TPU
You can't buy a TPU as a card. Google's docs list Compute Engine, Google Kubernetes Engine and the Gemini Enterprise Agent Platform as the ways to use them. Most people who use a TPU never choose one: they use a Google service that runs on them.
Frequently asked questions
What does TPU stand for?
TPU stands for Tensor Processing Unit. It is a custom chip Google designed to accelerate machine learning.
What is the difference between a TPU and a GPU?
A TPU is built around a matrix unit for neural network math, while a GPU is a more general parallel processor that began with graphics. Google's docs say GPUs fit better when a model has custom operations that must partly run on the CPU.
Can I buy a TPU?
Google's docs list Compute Engine, Google Kubernetes Engine and the Gemini Enterprise Agent Platform as the ways to use TPUs. Google has also begun deploying them in customers' own data centers.
What are TPU 8tx and TPU 8i Max?
They are two eighth-generation TPU systems Google announced on November 9, 2026. TPU 8tx targets secure, confidential inference, and TPU 8i Max packs two TPU 8i chips together for large context windows.
Sources
- TPU 8tx and TPU 8i MaxGoogle Cloud Blog
- Introduction to Cloud TPUGoogle Cloud
- In-Datacenter Performance Analysis of a Tensor Processing UnitarXiv


