Key takeaways
- A looped transformer runs a block of its layers several times per token with the same weights, adding computation without adding parameters.
- Inkling 2 Mini, released November 20, 2026, runs its middle 12 layers three times per token, giving 5.4B effective active parameters from 3.4B.
- Google Research found looped models nearly match deeper plain models on reasoning but do worse on memorizing facts.
A looped transformer is a language model that runs some of its layers more than once for each token, reusing the same weights on every pass. It gets the depth of a bigger model while storing the weights of a smaller one. The idea is years old, but it reached open-weight models this week: Thinking Machines Lab's Inkling 2 Mini, released on November 20, 2026, runs its middle 12 layers three times per token, and SenseNova-Loop-9B, released on November 17, runs 24 of its 32 layers twice.
How a looped transformer works
A normal transformer is a stack of layers, each with its own weights, and a token passes through each layer once. A 24-layer model does 24 layer computations per token and stores 24 sets of weights.
A looped transformer breaks that link. It feeds a block's output back into the same block several times before moving on. Each extra pass adds computation but no new parameters.
Most current designs split the model into three parts:
- An entry block that runs once and turns the input into the model's internal representation.
- A looped block in the middle that runs several times.
- An exit block that runs once and turns the result into the next token.
The recurrent-depth paper by Jonas Geiping and colleagues calls these the prelude, the recurrent block and the coda.
The looped block works on the model's hidden state, the internal numbers it keeps for each token. A chain-of-thought model spends more compute by writing more tokens. A looped model spends more per token without writing anything down. Researchers call this latent reasoning.
How Inkling 2 Mini and SenseNova-Loop-9B loop
| Inkling 2 Mini | SenseNova-Loop-9B | |
|---|---|---|
| Released | November 20, 2026 | November 17, 2026 |
| Distinct layers | 24 | 32 |
| Looped block | Middle 12 layers, run 3 times | Layers 4 to 27, run 2 times |
| Layer computations per token | 48 | 56 |
| Parameters | 26.4B total, 3.4B active (mixture of experts) | About 9B total |
| Effective size with looping | 5.4B active per token | About 14.6B "inference-equivalent" |
| License | Apache 2.0 | Apache 2.0 |
Inkling 2 Mini is also a mixture-of-experts model, which runs only part of its network for each token: each token goes to 8 of 128 experts per layer. The model card says each extra pass adds about 1.0B effective active parameters, which is how 3.4B becomes 5.4B.
Thinking Machines explains its choice of layers in the Inkling 2 Mini announcement. The company says the middle layers "carry more abstract reasoning," and reports that looping the middle 12 layers beat looping the last 12 on 27 of 28 evaluations. It puts the gain from middle-layer looping at 1.51 to 1.91 times in equivalent compute, against 1.24 to 1.36 times for looping the last layers. The benchmarks are in the Inkling 2 Mini release story.
SenseNova-Loop-9B uses two loops by default. Its team tested one to four loops and reports that more loops kept helping on some tasks: going from two to four loops added 1.04 points on LiveCodeBench V6. Neither model card describes a setting that lets users change the loop count, and the SenseNova page lists loops that adapt to each input as future work.
Where the idea comes from
Universal Transformers (2018). Mostafa Dehghani and colleagues at Google Brain and DeepMind proposed a transformer that applies the same layer repeatedly instead of stacking different ones. Their Universal Transformer also let each token decide when to stop, through a halting mechanism called Adaptive Computation Time. The authors showed that under certain assumptions the design is Turing-complete, which a fixed-depth transformer is not. On the bAbI reasoning tasks it cut average error to 0.29, from the standard transformer's 22.1.
Looped Transformers as Programmable Computers (2023). Angeliki Giannou and colleagues at the University of Wisconsin-Madison and Princeton showed what a loop can do in principle. They set a transformer's weights by hand, put it in a loop, and used the input as a punchcard holding a program and its memory. A looped 13-layer transformer could then run a calculator, matrix inversion and even a simple form of gradient descent. The weights were built, not trained, and the authors warn against assuming trained language models work the same way.
Recurrent depth at scale (2025). Geiping and colleagues trained a 3.5B-parameter looped model, called Huginn, on 800 billion tokens. The number of loops was drawn at random during training, so the model works with any count. At test time, more loops meant better answers: the abstract says performance kept improving up to a compute load equivalent to a 50-billion-parameter model. On the GSM8K math benchmark, an averaged version of the model reached 47.23% at 64 loops. It could also stop early on easy tokens with no extra training. The authors call it "still a proof-of-concept." The weights and the code are public.
What looping is good at, and what it isn't
The clearest test of the trade-off comes from a Google Research paper by Nikunj Saunshi and colleagues, Reasoning with Latent Thoughts.
The finding: a k-layer model looped L times nearly matches a model with k times L distinct layers on reasoning, but not on memorization. On a synthetic addition task, a 2-layer model looped 6 times scored 99.5, against 38.8 for the same model without the loop. At 1 billion parameters, a 12-layer model looped twice scored 34.3 on math word problems, above the 24-layer model's 29.3. The same looped model did worse than the 24-layer model on closed-book questions, 9.3 against 11.2, and had worse perplexity, a measure of how well a model predicts text: 7.90 against 7.40, where lower is better.
The authors describe this as an "inductive bias towards good reasoning." Reusing layers helps a model think in steps but leaves fewer places to store facts.
There is a cost on the other side. Looping saves memory, not compute. Inkling 2 Mini has 3.4B active parameters but does the work of about 5.4B per token, so it needs more compute per token than its active size suggests.
Thinking Machines says the trade pays off: a non-looped version of the model cost 7.5% less compute to train but needed 140% more inference compute to match the looped version's loss on 512,000-token agentic traces. That is the company's own measurement.
Why safety researchers watch it
Latent reasoning is harder to oversee. People can read a written chain of thought and catch problems. Reasoning inside a loop leaves nothing to read.
Thinking Machines says looping lets the model "reason longer without writing its reasoning down, which matters for monitoring." Its tests found no meaningful drop in how well the model's reasoning could be monitored "at this scale." That qualifier matters: Inkling 2 Mini runs three loops. Huginn ran up to 64 in testing, and the announcement reports no monitoring results for loop counts that high.
What to watch next
Both releases loop a fixed number of times. The research points to a bigger prize: models that loop more on hard tokens and less on easy ones, as the Universal Transformer and Huginn did in experiments. SenseNova lists that as future work. Watch for an open model with an adjustable loop count, and for independent benchmarks against plain models of the same speed.
Frequently asked questions
What is the difference between a looped transformer and a Universal Transformer?
The Universal Transformer, from 2018, is an early looped transformer: it applies one shared layer repeatedly and lets each token decide when to stop. Current looped models such as Inkling 2 Mini usually loop only a middle block, with a fixed number of passes, and run the first and last layers once.
What is the original looped transformer paper?
The idea of reusing transformer layers goes back to Universal Transformers by Dehghani et al. (2018). The term is most closely tied to Looped Transformers as Programmable Computers by Giannou et al. (2023), which showed a looped 13-layer transformer with hand-set weights can run programs such as a calculator and matrix inversion.
Is Inkling 2 Mini a looped transformer?
Yes. Thinking Machines Lab calls it its first released looped transformer. Its middle 12 layers run three times per token, for 48 layer computations from 24 layers of weights.
Does a looped transformer run faster?
No. Looping saves memory, not compute: each extra pass adds work per token. Inkling 2 Mini has 3.4B active parameters but does the work of about 5.4B per token.
Sources
- Inkling 2 Mini model cardThinking Machines Lab
- Inkling 2 MiniThinking Machines Lab
- SenseNova-Loop-9B model cardSenseNova
- Universal TransformersarXiv (Dehghani et al.)
- Looped Transformers as Programmable ComputersarXiv (Giannou et al.)
- Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth ApproacharXiv (Geiping et al.)
- Reasoning with Latent Thoughts: On the Power of Looped TransformersarXiv (Saunshi et al.)
- Huginn-0125Hugging Face
