# What is an energy-based model: AI that scores answers, not tokens

> An energy-based model rates how well an answer fits its input and searches for the best fit. A new paper uses the idea to solve 99.9% of hard Sudokus with a 1.8M-parameter model.

URL: https://claudegpt.si/news/what-is-an-energy-based-model  
Published: 2026-11-18T19:03:01.183Z  
Author: ClaudeGPT Si Newsroom  
Section: Research  
Topics: Energy-based models, Research, Reasoning, Yann LeCun  
Publisher: ClaudeGPT Si

## Key takeaways

- An energy-based model gives each possible answer an energy score, low for answers that fit the input, and finds its answer by searching for the lowest energy.
- The idea goes back to Hopfield networks and Boltzmann machines in the 1980s and was laid out in LeCun and colleagues' 2006 tutorial on energy-based learning.
- Energy-Based Reasoners, a November 2026 paper from Duke, UIUC and Harvard, reports 99.9% on Sudoku-Extreme with 1.8M parameters and up to 345 times fewer inference FLOPs than EqR.

An energy-based model is an AI model that, instead of producing an answer directly, gives every possible answer a score called energy: low for answers that fit the input, high for answers that don't. To get an answer, the model searches for the one with the lowest energy. The idea is decades old, and it is back in the news because of [Energy-Based Reasoners](https://energy-based-reasoners.github.io/), a paper from Duke, UIUC and Harvard that drew 259 points on the Hacker News front page on November 18, 2026.

That paper reports a 1.8-million-parameter model that solves 99.9% of a hard Sudoku benchmark with far less compute than earlier models of its kind.

## How an energy-based model works

The standard reference is [A Tutorial on Energy-Based Learning](http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf), written in 2006 by Yann LeCun and four colleagues at New York University. It defines the energy as "a measure of compatibility" between the variables a model looks at. Take a photo and a label. The pair (photo of a cat, "cat") should get low energy. The pair (photo of a cat, "truck") should get high energy.

Two jobs follow from that:

- **Inference** means answering a question. The model fixes what it knows (the photo), then looks for the value of what it doesn't know (the label) that makes the energy as low as possible.
- **Learning** means shaping the energy function so correct answers sit low and wrong ones sit high.

Picture hills and valleys over every possible answer. Training digs valleys where the right answers are. Inference rolls a ball downhill until it settles, often with gradient descent, the same slope-following used to train neural networks.

## How it differs from a probabilistic model

Most models output probabilities, which must add up to 1 across all possible answers. The tutorial points out that this requirement has a cost: computing that total, called the partition function, is often intractable when the space of answers is huge. Energy-based models skip it. They only need correct answers to score lower than wrong ones. The authors argue this leaves much more freedom in how a model is built and trained. Most probabilistic models are a special case of energy-based ones.

## How it differs from next-token prediction

A large language model is autoregressive: it writes its answer one token (a word or part of a word) at a time, committing to each before the rest exists. Reasoning models work around this by writing out long chains of thought before answering, so extra thinking means extra tokens. (See [what a reasoning model is](https://claudegpt.si/news/what-is-a-reasoning-model) for more on that approach.)

An energy-based model works on the whole answer at once. It can start from a rough guess, check how well the guess fits, and keep improving it. The energy also tells it when it's done: when the slope is flat, further steps won't help. And if it makes several attempts, it can pick the one with the lowest energy without a separate checker.

The catch: inference becomes a search, which can be slow or get stuck in the wrong valley.

## Where the idea comes from

Energy functions in neural networks go back to the 1980s. The [2024 Nobel Prize in Physics](https://www.nobelprize.org/prizes/physics/2024/press-release/) went to John Hopfield and Geoffrey Hinton partly for this work. The Royal Swedish Academy of Sciences describes how a Hopfield network stores images so that they have low energy. Given a damaged image, it updates its nodes so the energy falls, and settles on the stored image closest to the one it was fed. Hinton built on that with the Boltzmann machine, which used tools from statistical physics to learn patterns in data and generate new examples.

LeCun has kept making the case. In June 2023, [Meta described I-JEPA](https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/), an image model based on the joint-embedding predictive architecture LeCun proposed the year before. JEPA predicts an abstract representation of a missing part of an input, not its pixels. Meta's figure caption puts the goal in energy terms: assign "high energy to incompatible inputs" and low energy to compatible ones. The same thinking runs through many [world models](https://claudegpt.si/news/what-is-a-world-model), which predict what happens next in compressed form rather than frame by frame.

## What Energy-Based Reasoners do

The new paper, [posted to arXiv on November 6](https://arxiv.org/abs/2611.08977), is by William Chen and Zeyu Li of Duke University, Alexi Gladstone and Heng Ji of UIUC, and Yilun Du of Harvard. It applies the energy idea to looped models: small networks that run the same block over and over on an internal "scratchpad" state, getting more thinking from repetition rather than size.

The authors argue that existing looped models have no measure of progress, so they need separate machinery to decide when to stop and which answer to keep. An Energy-Based Reasoner (EBR) learns one energy function over its internal state and uses it three ways:

1. **Refine:** each loop is a gradient descent step down the energy.
2. **Stop:** a trajectory halts when the energy's slope falls below a threshold, so easy puzzles finish early.
3. **Select:** among several attempts, keep the lowest-energy one.

A training method they call state-based explorative modeling helps it learn several valleys when a puzzle has more than one answer.

## The results, as the authors report them

All numbers below are from the paper, and the baselines' scores are taken from their own papers, which follow their own evaluation rules.

- **Sudoku-Extreme** (9x9 puzzles; 1,000 training puzzles, 422,786 test puzzles): 99.4% solved with one attempt, against 97.9% for Looped Flows and 87.4% for TRM. With five attempts it reaches 99.9%, ahead of EqR's 99.8% with 128 attempts.
- **Compute:** stopping on a flat slope cuts inference FLOPs 10.8 times against a fixed number of loops, and at comparable accuracy EBRs use up to 345 times fewer FLOPs than EqR. The Hacker News headline says 385x. The paper and project page say 345x.
- **Maze-Hard** (30x30 mazes): 87.4% with one attempt, a small step past FPRM's 87.0%.
- **Multiple solutions:** on 10x10 N-Queens, 20 samples find 92.0% of the distinct valid solutions, against 57.5% for GRAM.

Each training run fits on one NVIDIA H200 GPU in 3 to 8 hours. In an ablation, swapping energy descent for an ordinary learned loop dropped one-attempt Sudoku accuracy from 98.96% to 60.05%.

## The limits

The authors list them. Training is more expensive per step, because descending an energy during training needs second-order derivatives; they wrote custom GPU kernels to cope. There are extra settings to tune, such as step size and noise. Gains on Maze-Hard were small, which they put down to its 1,000 training mazes. Most important, every test is a structured puzzle. Whether the approach scales to large models and to language is still an open question, they write.

So this is no sign that energy-based models will replace chatbots. It shows that on puzzles where an answer can be checked as a whole, scoring and refining can beat guessing forward, and cheaply.

The [code is on GitHub](https://github.com/wwillchen/ebr) under the MIT license, with datasets and trained checkpoints. The README says the team is preparing ARC-AGI results and will release those training configs and checkpoints later.

## Frequently asked questions

### What is the difference between energy-based models and probabilistic models?

A probabilistic model must output probabilities that add up to 1, which requires computing a normalizing total that is often intractable. An energy-based model only needs correct answers to score lower than wrong ones, and probabilistic models can be seen as a special case of it.

### What does Yann LeCun have to do with energy-based models?

LeCun co-wrote the 2006 paper A Tutorial on Energy-Based Learning, the standard reference on the subject. His joint-embedding predictive architecture (JEPA), which Meta used for I-JEPA in 2023, is framed in the same terms: low energy for compatible inputs, high energy for incompatible ones.

### Is there open-source code for energy-based models?

Yes. The Energy-Based Reasoners code is on GitHub under the MIT license, with scripts to build the Sudoku, maze, N-Queens and graph coloring datasets and download trained checkpoints.

### Are energy-based models used in large language models?

Not in the mainstream chatbots, which predict text one token at a time. The Energy-Based Reasoners authors tested only structured puzzles and say whether the approach scales to language models is still open.

## Sources

- [Energy-Based Reasoners: Your Recurrent Reasoner Should Be an Energy-Based Model (project page)](https://energy-based-reasoners.github.io/), Chen et al.
- [Energy-Based Reasoners: Your Recurrent Reasoner Should Be an Energy-Based Model](https://arxiv.org/abs/2611.08977), arXiv
- [EBR code repository](https://github.com/wwillchen/ebr), GitHub
- [A Tutorial on Energy-Based Learning](http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf), LeCun, Chopra, Hadsell, Ranzato, Huang (NYU)
- [The Nobel Prize in Physics 2024: press release](https://www.nobelprize.org/prizes/physics/2024/press-release/), The Royal Swedish Academy of Sciences
- [I-JEPA: The first AI model based on Yann LeCun's vision for more human-like AI](https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/), Meta AI
