Intelligence Index v5 results show effort widens gaps and open-weight models trail by 12 points

On October 27, 2026, Artificial Analysis announced Intelligence Index v5, the new edition of its model intelligence benchmark, without disclosing the specific evaluation changes or ranking results in the announcement itself. On the morning of October 28, the group released the details on X. Humanity's Last Exam was replaced in the Index by HLE-Diamond, a refined 1,000-question version produced by CAIS and Scale AI, split evenly between reasoning and knowledge questions. Claude Opus 5.5 (max) led it at 68%, followed by GPT-6 Astra (max) at 64%. Artificial Analysis also launched Terminal-Bench-Science, which tests whether agents can carry out scientific research workflows in a terminal; experts curate and review each task, and a task passes only if every one of its tests passes. GPT-6 Astra (max) led at 63%, with Claude Opus 5.5 (max) at 59%.

According to Artificial Analysis, v5 comprises nine evaluations and leaves significant room for progress, with no single model leading all of them: Claude Opus 5.5 took the top spot on three, while five other models led the remaining six, including a 46% on FrontierCode 2.0. Weighting is split evenly across Knowledge Work, Coding and Science at 25% each. Knowledge Work covers AA-Briefcase at 15%, AutomationBench-AA at 5% and GDP.pdf at 5%; Coding covers FrontierCode 2.0 at 15% and Terminal-Bench 4.0 at 10%; Science covers Terminal-Bench-Science at 15% and CritPt v2 hard.

Costs varied widely among the leading models: Claude Opus 5.5 (max) cost $10.04 per Intelligence Index task, GPT-6 Astra (max) $6.24 and GPT-6.1 Sol (max) $1.27, with GPT-6.1 Sol (max) scoring within 4 points of Claude Opus 5.5 (max) at roughly one eighth of the cost. Effort settings also mattered more in v5 than in v4.3: Claude Opus 5.5 scored 35 at low effort and 58 at max effort, at about 10x the cost per task ($0.98 vs $10.04), and Claude Sonnet 5.5's spread widened from 20 points in v4.3 to 29. v5 tasks also demand substantially more work, with every measured model generating more output tokens per task than in v4.3 and the median rising by two thirds, from 35k to 59k; agentic evaluations accounted for most of that usage.

Open weight models fell further behind. The leader, Kimi K3.1 Pro (weights not yet released), scored 46, 12 points behind the frontrunner, versus a 3-point gap in v4.3. Among models with released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) scored highest at 32. Later on October 28, discussion on the Reddit community LocalLLaMA raised objections: about 60% of the v5 Index rests on evaluations with private test sets or held-out answers, which makes reproducing the results impossible, and users questioned the accuracy of the ranking based on their own experience, citing GPT-Terra-6.1 placed above Fable-5.1. Publicly available information so far remains the breakdown and weights released by Artificial Analysis, with no further response to those criticisms.