Artificial Analysis reweights its Intelligence Index to v5, widening the apparent open/closed capability gap
2026-10-28
Artificial Analysis released version 5 of its widely-used Intelligence Index, restructured into four equally-weighted pillars (Knowledge Work, Coding, Science, General) built from new and updated sub-benchmarks including AA-Briefcase, FrontierCode 2.0, Terminal-Bench 4.0, Terminal-Bench-Science, CritPt v2, AA-Omniscience v1.1, and HLE-Diamond. Under the new weighting, eight closed models now score above the best open-weight model (Kimi K3.1 Pro). AI researcher Nathan Lambert described it as tuned toward "a general agentic collaborator" that "grows the gap to open models" relative to the prior version, and noted signs of strain (e.g., GPT-6.1 Terra outscoring Claude Fable 5.1).
Significance 4: A methodology change to the single most-cited independent intelligence benchmark, one that materially shifts the measured open/closed gap used as evidence in multiple tracked verdicts, meets the rubric's bar for moving a tracked axis even without a new model release.
Implications · machine-drafted, not owner judgment
Because the hard-problems and daily-driver verdicts both cite the Artificial Analysis Intelligence Index as evidence, a reweighting that changes which models look strongest changes the ground those verdicts stand on even if the underlying models haven't changed. It's a reminder, flagged by the benchmark's own critics, that single-number intelligence indices are judgment calls dressed as measurement.
- Whether the v5 ranking holds up against independent spot-checks or community pushback on specific weighting choices
- Whether the widened open/closed gap narrows again once open labs optimize against the new sub-benchmarks
Sources
- curator Artificial Analysis v5 and the benchmarker's dilemma retrieved 2026-10-30
Affected axes
Hard reasoning