# Artificial Analysis Intelligence Index: what it measures in v5

> The index averages nine tests in four equal categories, and 60% of it rests on private data. Its own pages list what it cannot tell you.

URL: https://claudegpt.si/news/artificial-analysis-intelligence-index-explained  
Published: 2026-10-28T10:55:58.829Z  
Author: ClaudeGPT Si Newsroom  
Section: Research  
Topics: Artificial Analysis, Benchmarks, Language models, Evaluation  
Publisher: ClaudeGPT Si

## Key takeaways

- Version 5.0, published October 27, 2026, averages nine evaluations in four categories worth 25% each.
- Artificial Analysis says private or held-out tests make up 60% of the index.
- Scores do not compare across versions, and effort settings change a model's score a lot.

The Artificial Analysis Intelligence Index is a single score that ranks AI language models by averaging results on nine tests. Artificial Analysis, an independent benchmarking company, published version 5 on October 27, 2026, and describes it in its [v5 announcement](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5) and its [methodology page](https://artificialanalysis.ai/methodology/intelligence-benchmarking). Here is what goes into it, how it is scored, and where its own authors say it stops being useful.

We covered the release itself in [our report on Intelligence Index v5](https://claudegpt.si/news/artificial-analysis-intelligence-index-v5). This article is about the index as a method.

## What the Intelligence Index includes

Version 5.0 has four categories, each worth 25% of the score. Each category holds two or three evaluations, according to the methodology page.

| Category | Evaluation | Weight in the index |
| --- | --- | --- |
| Knowledge Work | AA-Briefcase v1.1 | 15% |
| Knowledge Work | AutomationBench-AA | 5% |
| Knowledge Work | GDP.pdf | 5% |
| Coding | FrontierCode 2.0 | 15% |
| Coding | Terminal-Bench 4.0 | 10% |
| Science | Terminal-Bench-Science 0.1 | 15% |
| Science | CritPt v2 (hard subset) | 10% |
| General | AA-Omniscience v1.1 | 15% |
| General | HLE-Diamond | 10% |

Artificial Analysis says the index is primarily text-based and English-language. Image, speech and multilingual performance are measured separately.

Version 5 added FrontierCode 2.0, which has 132 private coding tasks graded on whether a maintainer would merge the change, and Terminal-Bench-Science, which has 70 science tasks done in a terminal. It removed GDPval-AA v2.1, SciCode and AA-LCR v1.1. Private or held-out tests now make up 60% of the index, the announcement says, which is meant to make it harder for models to be tuned to the questions.

## How the score is calculated

The index is a weighted average of the nine evaluations, using the weights above. Each test is scored in its own way:

- Test suites for code and terminal tasks are scored pass or fail, as pass@1, meaning the model gets credit if its first attempt passes.
- FrontierCode 2.0 uses tests plus a rubric graded by an LLM judge. Failing a blocking check scores zero.
- AutomationBench-AA scores zero if the model violates a guardrail.
- AA-Briefcase combines Elo ratings from pairwise comparisons with rubric grading. The Elo is frozen when a model is added, and it is anchored to GPT-5.5 (medium) at 1000.
- AA-Omniscience counts accuracy (10% of the index) and the share of answers that are not hallucinations (5%) separately.
- HLE-Diamond is graded by an LLM equality checker.

Several tests run more than once. GDP.pdf and CritPt v2 run five times, and the coding and science terminal tasks run three times, with scores aggregated across repeats.

Run settings are published too. Non-reasoning models run at temperature 0 and reasoning models at 0.6, unless the lab recommends otherwise. Non-reasoning models get up to 16,384 output tokens, and reasoning models get the maximum their makers disclose. Failed API calls are retried up to 30 times, and results hit by persistent failures are not published.

## Limits Artificial Analysis states

Artificial Analysis says the index, like any metric, "has limitations and may not apply directly to every use case." Its pages list specific ones:

- **Versions do not compare.** Scores are not directly comparable across versions because the tests and weights changed. A 55 in v5 is not a 55 in v4.3.
- **Settings change scores.** In the announcement, Claude Opus 5.5 scores 35 at low effort and 58 at max effort. A model's score depends on how it was run.
- **No model wins everything.** No single model leads all nine evaluations. Claude Opus 5.5, the top model at 58, leads three.
- **Some tests were trimmed.** CritPt v2 is a 40-question subset. Thirty of 70 questions were removed because seven frontier models solved them in all five attempts at launch.
- **Test versions differ from outside versions.** HLE-Diamond is not directly comparable to the full Humanity's Last Exam, GDP.pdf is not directly comparable to Surge AI's implementation, and AA-Omniscience v1.1 scores average 2 points lower than v1.
- **Judges are models.** Many of the tests are graded by LLM judges. The methodology page describes the prompts and checks it uses.
- **Reference values can move.** Artificial Analysis says it may update the AA-Briefcase normalization parameters.
- **Intervals are not given for the index.** The methodology page gives 95% confidence intervals for the Elo ratings in some knowledge-work tests, but states none for the index score.

## Is it reliable?

The pages do not make a claim of reliability, and we have not tested the index ourselves. What they do show is a method with published weights, settings and known gaps. The company also says 60% of the index rests on private data, which limits one known problem with public benchmarks.

The practical reading follows from the limits above. Compare models within one version of the index and at stated effort settings. Treat a gap of a point or two with caution, since no interval is published. And check the individual evaluations, because a model that scores well overall may not lead in the category you care about.

Artificial Analysis also launched a separate Progress Index v1 alongside v5. The announcement says the Intelligence Index "remains our primary measure of current model intelligence."

## Frequently asked questions

### What is the Artificial Analysis Intelligence Index?

It is a composite score from Artificial Analysis that averages a model's results on nine evaluations across knowledge work, coding, science and general knowledge. Version 5.0 was published on October 27, 2026.

### Is Artificial Analysis reliable?

Its methodology page publishes weights, run settings and known limits, and says 60% of the index uses private data. It states no confidence interval for the index score, so small gaps should be read with caution.

### Can I compare Intelligence Index scores across versions?

No. Artificial Analysis says scores are not directly comparable across versions because the tests and weights changed.

### Does the Intelligence Index cover images or other languages?

No. The index is primarily text-based and English-language. Artificial Analysis measures image, speech and multilingual performance separately.

## Sources

- [Artificial Analysis Intelligence Index v5](https://artificialanalysis.ai/articles/artificial-analysis-intelligence-index-v5), Artificial Analysis
- [Intelligence Benchmarking methodology](https://artificialanalysis.ai/methodology/intelligence-benchmarking), Artificial Analysis