# Cerebras splits inference between GPUs and its chips for 5x tokens per watt

> GPUs read the prompt and Cerebras systems write the answer; AWS and Fireworks AI have early access.

URL: https://claudegpt.si/news/cerebras-disaggregated-inference  
Published: 2026-11-18T06:58:50.751Z  
Author: ClaudeGPT Si Newsroom  
Section: AI Tools  
Topics: Cerebras, AI inference, AI chips, AMD  
Publisher: ClaudeGPT Si

## Key takeaways

- Cerebras now runs disaggregated inference in production, with GPUs doing prefill and its CS-3 systems doing decode.
- Cerebras says a GPT-OSS-120B endpoint served five times more tokens per megawatt than CS-3 systems alone, with no loss in speed.
- AWS, Fireworks AI, Gimlet Labs and General Compute have early access.

Cerebras has put disaggregated inference into production, the chipmaker said in a [blog post on November 17, 2026](https://www.cerebras.ai/blog/cerebras-launches-disaggregated-inference). The setup splits each request between two kinds of hardware: GPUs read the prompt, and Cerebras's wafer-scale systems write the answer. Cerebras says this delivers five times more tokens per megawatt with no loss in output speed.

The idea rests on how language models work. Answering a request has two phases. Prefill reads the whole prompt at once, which suits GPUs. Decode writes the reply one token (a word or part of a word) at a time, and its speed depends on how fast the chip can move data in memory. Cerebras's chips hold far more fast on-chip memory than a GPU, so it gives them the decode step and lets GPUs do the rest. Its software passes the model's working state, the key-value cache, between the two machines over a fast network link.

The 5x figure comes from Cerebras's own measurement. It compares a shared GPT-OSS-120B endpoint on the Cerebras Inference Cloud, using AMD Instinct GPUs for prefill and CS-3 systems for decode, against CS-3 systems alone, with data from November 14. No independent test has checked it yet, and the post gives no prices.

That matters more than raw speed. Data center power is now the main limit on how much AI companies can serve, so more tokens per megawatt means more customers on the same power budget. Cerebras says it plans to roll disaggregated inference out across hundreds of megawatts in its cloud and partner sites by the end of 2027.

Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute have early access. Partners can run the full stack, with vLLM or SGLang for prefill, or use a decode-only kit with their own prefill engine. For the basics of the two phases, see our explainer on [AI inference](https://claudegpt.si/news/what-is-ai-inference).

## Sources

- [Cerebras launches disaggregated inference](https://www.cerebras.ai/blog/cerebras-launches-disaggregated-inference), Cerebras
