# What is AI hallucination? Why models make things up

> A confident, false answer from a language model. Research from Anthropic and a 2025 academic paper explains why it happens and what reduces it.

URL: https://claudegpt.si/news/what-is-ai-hallucination  
Published: 2026-11-07T22:51:43.550Z  
Author: ClaudeGPT Si Newsroom  
Section: Research  
Topics: AI hallucination, Anthropic, Claude, Language models  
Publisher: ClaudeGPT Si

## Key takeaways

- An AI hallucination is output that is false or unsupported by the given context, stated with confidence.
- A September 2025 paper argues models hallucinate because training and benchmarks reward guessing over saying 'I don't know'.
- Anthropic's docs list fixes such as allowing 'I don't know', quoting sources first and verifying claims, but say none eliminates hallucinations.

An AI hallucination is a confident answer from a language model that is false or not backed by the text it was given. Anthropic's documentation defines it as output that is ["factually incorrect or inconsistent with the given context"](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations). The model isn't lying. It produces the most plausible-sounding continuation, and sometimes that continuation is wrong.

Two published explanations cover why this happens. A 2025 paper says models are trained and graded in ways that reward guessing. Anthropic's interpretability work says the model's "I don't know" reflex can fail to switch on. Both are below, along with what you can do about it.

## What an AI hallucination looks like

The common cases are a made-up fact stated plainly: a wrong birthday, an invented citation, a feature a product doesn't have. The same word covers a summary that adds details the source document never contained.

Two examples from the research papers:

- Kalai and colleagues note that a model asked for someone's birthday has to guess when that date appeared rarely or once in its training data. For a given person, there are 364 wrong dates for every right one.
- In Anthropic's experiments on Claude 3.5 Haiku, the model declined to answer about an unknown name, "Michael Batkin." When researchers switched on its "known answer" features by hand, it consistently claimed that Batkin plays chess.

## Why language models hallucinate

### Pretraining: some facts can't be learned from patterns

The paper ["Why Language Models Hallucinate"](https://arxiv.org/abs/2509.04664) (Kalai, Nachum, Vempala and Zhang, September 4, 2025) treats hallucination as an ordinary classification error. A model that can't tell a false statement from a true one will sometimes produce the false one.

Arbitrary facts are the clearest case. A birthday follows no pattern, so the model can only get it right if it saw it. The authors link the minimum error rate to the "singleton rate", the share of facts that appear exactly once in the training data. Their example: if 20% of birthday facts appear once, a base model should be expected to get at least about 20% of them wrong.

Some errors come from the model's shape instead. The paper points to letter counting, where DeepSeek-V3 gave inconsistent answers while the reasoning model DeepSeek-R1 spelled the letters out and counted them correctly.

### Post-training and benchmarks: guessing scores better than abstaining

The paper's second argument is about grading. Most major benchmarks, including GPQA, MMLU-Pro, SWE-bench and HLE, score answers as right or wrong. A blank or "I don't know" earns zero, the same as a wrong answer. Under that rule, guessing never costs a model anything, so a model that always guesses beats one that admits uncertainty on the leaderboard.

The authors call this an "epidemic" of penalizing uncertainty. Their proposed fix is to change how the existing benchmarks are scored. Tell the model the confidence threshold in the prompt, for example answer only if you are more than 75% sure, and take two points off for a wrong answer. Then a guess pays only when the model is confident enough. This is a proposal from the paper, not something every lab has adopted.

### Inside the model: the "don't know" circuit can misfire

Anthropic's [March 27, 2025 research post](https://www.anthropic.com/research/tracing-thoughts-language-model) looked inside Claude 3.5 Haiku. It reports that declining to answer is the default: a circuit that is normally on makes the model say it lacks the information. When the model recognizes a name like Michael Jordan, a "known entities" feature inhibits that default and it answers.

Hallucinations can happen when the model recognizes a name but knows nothing else about the person. The known-entity feature still fires, the "don't know" feature is suppressed, and the model fills in something plausible. This is one lab's finding on one model, but it matches the paper's point that the failure is a misjudgment about what the model knows.

## How to reduce AI hallucinations

Anthropic's guide to [reducing hallucinations](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations) lists these techniques (read on November 7, 2026):

| Technique | What to do |
| --- | --- |
| Allow "I don't know" | Tell the model explicitly it may say it is unsure or that the material lacks the answer. |
| Quote first | For long documents (over 20,000 tokens), ask for word-for-word quotes before the analysis, then base the answer only on those quotes. |
| Cite and verify | Have the model cite a quote for each claim, then check each claim against the source. If it can't find support, it must retract the claim. |
| Restrict outside knowledge | Instruct it to use only the documents provided, not its general knowledge. |
| Best-of-N | Run the same prompt several times. Differing answers flag a likely hallucination. |
| Iterative refinement | Feed the output back and ask the model to verify or expand its earlier statements. |
| Review the reasoning | With extended thinking, read the summarized thinking blocks when an answer looks wrong to spot faulty logic. |

Anthropic's own caveat applies to all of them: they "significantly reduce" hallucinations but don't eliminate them, so verify anything high-stakes.

Retrieval, where the model looks up documents before answering, supports the "only use these sources" approach. We explain it in [what is RAG](https://claudegpt.si/news/what-is-rag-retrieval-augmented-generation).

For everyday use, the same ideas translate directly. Paste in the source you want summarized. Ask for the quote behind each claim. Ask the same question twice and compare. Treat names, numbers, dates and citations as the parts to check first, since those are the arbitrary facts the paper says models are worst at.

## What we don't know

The paper's authors argue that hallucination is a predictable result of training and grading, not a bug waiting for a patch. A model that abstains more may look worse on today's leaderboards, which is the incentive problem the paper describes. Whether labs will change their scoring is open. And Anthropic's circuit findings come from one small model, so they shouldn't be read as a full account of how every model works.

## Frequently asked questions

### What does AI hallucination mean?

It means a language model produces text that is factually wrong or not supported by the material it was given, usually in a confident tone. Anthropic describes it as output that is factually incorrect or inconsistent with the given context.

### Can AI hallucinations be eliminated?

Not with current methods. Anthropic's guide says its techniques significantly reduce hallucinations but don't eliminate them, and the 2025 paper argues some errors are statistically unavoidable for facts that appear rarely in training data.

### What are examples of AI hallucinations?

A wrong birthday for a person the model saw little about, an invented citation, or a summary that adds details the source lacks. In Anthropic's experiment, Claude was pushed to claim an unknown person named Michael Batkin plays chess.

### How do I check an AI answer for hallucinations?

Ask for a supporting quote for each claim and check it against the source, or run the same question several times and compare. Check names, numbers, dates and citations first.

## Sources

- [Reduce hallucinations](https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations), Anthropic
- [Tracing the thoughts of a large language model](https://www.anthropic.com/research/tracing-thoughts-language-model), Anthropic
- [Why Language Models Hallucinate (arXiv:2509.04664)](https://arxiv.org/abs/2509.04664), arXiv