Key takeaways
- Test time compute is the computing a model spends while answering, used for longer reasoning, sampling many answers and voting, or searching through solution steps.
- OpenAI's o1 scored 74% on AIME 2024 with one sample and 93% when 1,000 samples were re-ranked, against 12% for GPT-4o.
- Gains flatten: Snell et al. found that on the hardest problems a bigger pretrained model was the better use of compute, and s1's budget forcing levels off.
Test time compute is the computing a language model spends while it answers a question, as opposed to the computing spent training it. Spending more of it means letting the model write a longer chain of reasoning, sample many answers and pick the best, or search through possible solution steps. Since OpenAI's o1 in September 2024, it has become the main way labs make models better at math, code and other hard problems without making them bigger.
The idea is simple. A person given an hour on a hard problem does better than one given ten seconds. For years, language models got one pass: read the question, write the answer, done. The research below shows that letting a model think longer, or try more times, buys accuracy that would otherwise take a much larger model.
Three ways to spend test time compute
Think longer. The model writes out intermediate steps before the answer, a chain of thought. Reasoning models such as o1 and DeepSeek-R1 are trained to do this at length, often for thousands of tokens, checking and revising their own work along the way.
Try many times and pick. The model answers the same question many times, and something chooses among the answers. The simplest chooser is a majority vote. Others are a separate scoring model, called a verifier or reward model, or an automatic check such as running unit tests on code.
Search. Instead of whole answers, the system grows partial solutions step by step, scores each step with a verifier and keeps the promising branches, much like a chess engine looking ahead.
All three turn extra computation at answer time into accuracy. They differ in what they need: thinking longer needs a model trained to use the extra tokens well, voting needs answers that can be compared, and search needs a verifier you can trust.
The evidence, paper by paper
Self-consistency (2022). Xuezhi Wang and colleagues at Google sampled several reasoning paths from a model and took the most common final answer, instead of trusting the single most likely path. Across four models up to PaLM-540B, this beat ordinary chain-of-thought prompting on every task they tried, including gains of 17.9 points on the GSM8K math word problems and 12.2 on AQuA.
Large Language Monkeys (July 2024). Bradley Brown and colleagues simply sampled more. The share of problems solved by at least one sample kept rising across four orders of magnitude of samples, roughly in a straight line on a log scale. On SWE-bench Lite, a set of real GitHub issues, DeepSeek-Coder-V2-Instruct fixed 15.9% with one try and 56% with 250 tries, above the 43% that the best single-try system had at the time. The catch: that only pays off when you can check answers automatically, as with code tests. Without a checker, majority voting and reward models stopped improving beyond several hundred samples.
Snell et al. (August 2024). Charlie Snell, Jaehoon Lee, Kelvin Xu and Aviral Kumar asked how to spend a fixed test time budget well on the MATH competition benchmark. The best method depended on how hard the question was. Letting the model revise its own answer worked best on easier questions; searching with a step-by-step verifier worked best on harder ones. Choosing the method per question was more than 4 times as efficient as plain best-of-N sampling. With the total computation held equal, a small model with extra test time compute beat a model 14 times larger, but only on problems the small model could already sometimes solve.
OpenAI o1 (September 12, 2024). OpenAI's o1 announcement said the model's performance improves with more reinforcement learning during training and with "more time spent thinking (test-time compute)". On the 2024 AIME math exam, GPT-4o averaged 12%. o1 scored 74% with one sample, 83% taking the consensus of 64 samples, and 93% when 1,000 samples were re-ranked by a scoring function. That stacked both kinds of test time compute: long reasoning inside each answer, then many answers.
DeepSeek-R1 (January 2025). DeepSeek showed that reinforcement learning alone can teach a model to think longer. As training went on, DeepSeek-R1-Zero's answers grew steadily longer without anyone telling it to, and its AIME 2024 score rose from 15.6% to 77.9%, then to 86.7% with majority voting on top. The paper describes an "aha moment" when the model suddenly started using the word "wait" to go back and recheck its work.
s1 (January 2025). Niklas Muennighoff and colleagues looked for the simplest recipe. They fine-tuned Qwen2.5-32B-Instruct on just 1,000 worked examples and added "budget forcing": cutting the model's thinking off to save compute, or appending "Wait" when it tries to stop so it keeps checking. Forcing more thinking lifted its AIME 2024 score from 50% to 57%. Model, data and code are open source.
Why it became the big scaling lever
Making models better used to mean training bigger ones on more data, which costs more every round. Test time compute adds a second dial. A lab can train a model to reason, then sell more or less thinking per request. A developer can spend heavily on the one hard question and almost nothing on the easy ones.
That dial is now a standard API setting. As of November 22, 2026, OpenAI's reasoning guide lists a reasoning.effort parameter with model-dependent values from none to max, and says lower effort "favors speed and lower token usage." Anthropic's effort docs list low, medium, high, xhigh and max for Claude models, and its older fixed thinking budget, budget_tokens, is deprecated on newer models.
The costs and limits
Money. Thinking tokens are billed. OpenAI says reasoning tokens are billed as output tokens even though the API does not show them, and Anthropic reports thinking tokens in its usage data for the same reason. Sampling 64 answers costs roughly 64 answers.
Latency. Longer thinking means a longer wait. Anthropic's docs say higher thinking budgets come "at the cost of increased latency" and recommend batch processing above 32,000 thinking tokens to avoid timeouts.
Diminishing returns. Every curve above bends. Snell et al. found that on the hardest MATH questions no method made much progress, and that for hard questions spending the compute on pretraining a bigger model was the better trade. The s1 authors report that budget forcing flattens out, and that suppressing the end of thinking too often sends the model into repetitive loops. Anthropic's docs say that for Claude Opus 4.7, max effort adds significant cost for relatively small quality gains on most workloads.
Overthinking. More thinking can hurt. Snell et al. saw search make easy questions worse by gaming the verifier. DeepSeek says R1 still overthinks simple questions, and Anthropic warns that max effort can lead to overthinking on some structured-output tasks.
The practical lesson from the papers matches the API advice: test time compute is a budget, not a free upgrade. Spend it on problems that are hard but within reach, where answers can be checked, and measure whether a higher setting actually helps on your own tasks before paying for it.
Frequently asked questions
What does test time compute mean in LLMs?
It is the computation a language model uses at inference, when it answers a question, rather than during training. Spending more of it means longer reasoning, sampling many answers and choosing one, or searching over solution steps.
What is test time compute scaling?
It is the finding that accuracy keeps rising as a model gets more computation at answer time. OpenAI said o1 improves with more time spent thinking, and Brown et al. found problems solved rose roughly log-linearly over four orders of magnitude of samples.
Is test time compute the same as chain of thought?
No. Chain of thought, writing out steps before the answer, is one way to spend test time compute. Sampling many answers and voting, or searching with a verifier, are others, and they can be combined.
Which paper is the main test time compute paper?
The most cited starting points are Snell et al.'s August 2024 paper, Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (arXiv 2408.03314), and Wang et al.'s 2022 self-consistency paper (arXiv 2203.11171).
Sources
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model ParametersarXiv (Snell et al.)
- Self-Consistency Improves Chain of Thought Reasoning in Language ModelsarXiv (Wang et al.)
- Large Language Monkeys: Scaling Inference Compute with Repeated SamplingarXiv (Brown et al.)
- Learning to reason with LLMsOpenAI
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement LearningarXiv (DeepSeek-AI)
- s1: Simple test-time scalingarXiv (Muennighoff et al.)
- Reasoning modelsOpenAI
- EffortAnthropic
- Extended thinkingAnthropic

