# What is mixture of experts? A 594B model that uses 17B per token

> MoE models store hundreds of billions of parameters but run only a few percent of them per token. Here is how that works, from the papers to Inkling 2 Small and Kimi K3.1 Pro.

URL: https://claudegpt.si/news/what-is-mixture-of-experts  
Published: 2026-11-06T02:54:16.045Z  
Author: ClaudeGPT Si Newsroom  
Section: Research  
Topics: Mixture of experts, Thinking Machines Lab, Inkling, Kimi, Language models  
Publisher: ClaudeGPT Si

## Key takeaways

- A mixture-of-experts model routes each token to a few of its many experts, so only a fraction of its parameters run per token.
- Inkling 2 Small has 594B total and 17B active parameters, about 3%; Kimi K3.1 Pro has 2.8T total and 104B active.
- MoE saves compute per token, but all experts must still be stored, so memory is not reduced.

A mixture-of-experts (MoE) model is a neural network split into many smaller sub-networks, called experts, where a small router picks only a few of them for each token of text. The model holds a large number of parameters but uses a fraction of them per word. That is how Thinking Machines Lab's [Inkling 2 Small](https://thinkingmachines.ai/news/inkling-2-small/), released on November 5, 2026, can have 594 billion parameters in total and 17 billion active at a time.

So when you see "594B total, 17B active" in a model announcement, the second number is closer to what each token costs to compute. The first number is closer to how much the model can store.

## How a mixture-of-experts model works

In an ordinary ("dense") model, every parameter takes part in every calculation. In an MoE model, some of the layers are replaced by a bank of experts. Each expert is a small feed-forward network. A router, which is a tiny trained network, looks at each token and scores the experts. Only the top few run. Their outputs are combined and passed on.

Three terms come up in every announcement:

- **Total parameters:** everything the model contains, all experts included.
- **Active parameters:** the ones used for a single token. This sets the compute per token.
- **Top-k routing:** how many experts the router picks per token. Mixtral picks 2.

The experts are not hand-assigned subjects like "the math expert". The router and the experts are trained together, and which expert handles what is learned, not designed.

## Where the idea comes from

Three papers explain most of what you will read about MoE today.

**Shazeer et al., 2017.** [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://arxiv.org/abs/1701.06538) put a layer with up to thousands of feed-forward experts between stacked LSTM layers, with a trainable gating network choosing a sparse subset for each input. The authors report more than a 1000-fold gain in model capacity with only minor losses in computational efficiency, and a model of up to 137 billion parameters in the MoE layer. They applied it to language modeling and machine translation.

**Fedus, Zoph and Shazeer, 2021.** [Switch Transformers](https://arxiv.org/abs/2101.03961) simplified the routing to cut complexity, communication and compute costs, and added techniques to make sparse models train stably, including in bfloat16, a lower-precision number format. The paper reports up to 7x faster pre-training than T5-Base and T5-Large with the same compute, and trillion-parameter models with a 4x speedup over T5-XXL.

**Jiang et al., 2024.** [Mixtral of Experts](https://arxiv.org/abs/2401.04088) made the idea concrete for open models. Mixtral 8x7B has 8 experts in each layer, and the router picks 2 per token at every layer. It has about 47 billion parameters but uses about 13 billion per token. Its authors report that it matches or beats Llama 2 70B and GPT-3.5 on all the benchmarks they evaluated. Both versions were released under Apache 2.0.

## MoE in this week's models

Recent releases show how far the ratio has moved.

| Model | Total parameters | Active per token | Active share |
| --- | --- | --- | --- |
| Mixtral 8x7B (January 2024) | about 47B | about 13B | about 28% |
| Inkling 2 Small (November 5, 2026) | 594B | 17B | about 3% |
| Kimi K3.1 Pro | 2.8T | 104B | about 4% |

The shares are our arithmetic from the published numbers.

Thinking Machines says Inkling 2 Small [is a mixture-of-experts model](https://thinkingmachines.ai/news/inkling-2-small/) with 594 billion total and 17 billion active parameters, with image input and variable effort. Its post does not say how many experts it has or how it routes. It is available now through Tinker with closed weights. The company says it intends to release full weights under Apache 2.0 "next week," after external safety testing and red-teaming. That is a plan, not a release.

Moonshot's Kimi K3.1 Pro, which [we covered here](https://claudegpt.si/news/kimi-k3-1-pro-release-date), has 896 experts, of which 16 are chosen per token, plus 2 shared experts, according to its [Hugging Face model card](https://huggingface.co/moonshotai/Kimi-K3.1-Pro). Its context length is 1,048,576 tokens. Its license is a custom one, not a standard open-source license.

## What MoE does and doesn't save

The saving is in compute per token. A model that activates 17 billion parameters does roughly the arithmetic work of a 17-billion-parameter dense model for each token, while drawing on far more stored knowledge. The Switch Transformer and Mixtral results above are the evidence: faster training for the same compute, and a small active footprint matching much larger dense models.

It does not make the model small. Every expert has to be available, because the router can pick any of them for the next token. So the full 594 billion parameters still have to live somewhere, which means memory, not arithmetic, is usually the bill you pay. That is our reading of how the numbers work, not a figure from the announcement. Check what a provider charges per token and what hardware the open weights need before you assume a low active count means a cheap deployment.

Two more caveats. Training sparse models took extra work: the Switch paper had to add techniques for stability. And a vendor's active-parameter count tells you little about quality. Compare models on benchmarks and on your own tasks.

## What to watch

Thinking Machines says Inkling 2 Small's weights are due next week. When they are out, the file sizes and the model card should show what the 594 billion total parameters cost to host, and whether the expert count and routing, which the announcement leaves out, are published.

## Frequently asked questions

### What are active parameters in an MoE model?

They are the parameters used to process a single token. In an MoE model the router picks a few experts per token, so the active count is much smaller than the total. Inkling 2 Small has 17B active of 594B total.

### Is a mixture-of-experts model faster than a dense model?

Per token it needs less compute, because only some experts run. The Switch Transformer paper reports up to 7x faster pre-training than T5-Base and T5-Large with the same compute. The full model must still fit in memory.

### What does top-2 routing mean?

The router chooses two experts for each token at each layer. Mixtral 8x7B works this way: 8 experts per layer, 2 used per token.

### Are experts specialized by topic?

Not by design. The router and experts are trained together, and which expert handles which token is learned during training rather than assigned by hand.

## Sources

- [Inkling 2 Small](https://thinkingmachines.ai/news/inkling-2-small/), Thinking Machines Lab
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://arxiv.org/abs/1701.06538), arXiv
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://arxiv.org/abs/2101.03961), arXiv
- [Mixtral of Experts](https://arxiv.org/abs/2401.04088), arXiv
- [moonshotai/Kimi-K3.1-Pro model card](https://huggingface.co/moonshotai/Kimi-K3.1-Pro), Hugging Face