LLM benchmarks & leaderboards

How models are ranked across the llm-provider landscape. The 2026 best practice (llm-leaderboard-stats) is a composite, multi-axis score, not a single number.

The axes

Capability benchmarks — GPQA Diamond, SWE-Bench Verified, MMLU, AIME, HumanEval, coding-arena.
Speed — output throughput (tokens/sec), time-to-first-token.
Price — per-token cost (llm-api-pricing).
Context window and agentic/coding performance.
Aggregators referenced across sources: Artificial Analysis Index, llm-stats.com.

Anatomy of a composite — the Artificial Analysis Intelligence Index

The clearest worked example of “composite, not a single number.” As of v4.1 (current, 2026-06) the artificial-analysis Intelligence Index is a weighted average of 9 evals across four categories, and the weighting is itself a thesis about what matters in 2026 artificial-analysis:

Agents — 34% (the biggest slice): GDPval-AA v2 (220 agentic file-output tasks), τ³-Banking (97 dual-control agent-user tasks).
Coding — 24%: Terminal-Bench v2.1 (89 terminal tasks), SciCode (288 code subproblems).
Scientific reasoning — 24%: Humanity’s Last Exam (2,158 open questions), GPQA Diamond (198 MC), CritPt (70 physics challenges).
General — 18%: AA-LCR (100 long-context-reasoning questions), AA-Omniscience (6,000 knowledge questions, split 8% accuracy + 4% non-hallucination rate).

Two things follow. Agents+coding = 58% of the score, so the index now rewards doing tasks over recalling facts; and the disclosed ±1% 95% confidence interval (from >10 repeats) is what lets it function as the reproducible yardstick the synthesis leans on. The weighting is a defensible editorial choice, not ground truth — the standing caveat on every leaderboard.

2026 snapshot (volatile — churns weekly)

Reasoning leaders are proprietary — Anthropic (Claude Mythos Preview, claude-opus-4-8 #2) and OpenAI (GPT-5.5 #3); Alibaba’s Qwen3.7 Max is the cheapest top-10 ($1.53/M); xAI’s Grok 4 Fast has the largest context (2.0M); Mercury 2 is fastest (784 tok/s). Among open weights, deepseek V4 Pro tops the Artificial Analysis Index; Qwen3.6 hits 77.2% SWE-Bench.

The other kind — novelty benchmarks

Everything above measures a model on tasks whose shape is known in advance, which is what makes them reproducible and what makes them gameable. ARC-AGI is built the other way: it scores tasks the model never saw in training, and its current version (ARC-AGI-3) makes the model infer the rules of an interactive game rather than answer a question opus-5-arc-agi-3. Two rules follow from that design and both are unusual here.

Model alone, no harness. ARC Prize excludes scaffolding software from official scores, so an ARC-AGI-3 number is a floor on what the same model does inside an agent. Every index in this page’s table measures the opposite: whatever the vendor ships.
Scores are far lower and move in jumps. The ARC-AGI-3 record went 7.8% → 30.2% in one model generation, where composite indices creep. A benchmark nobody has saturated discriminates; one at 90%+ mostly measures the noise floor, which is where ARC-AGI-1 and -2 now sit.

The catch is that a novelty benchmark stops measuring novelty once labs can aim at it, and the Opus 5 result is the live test case: a ~4× jump on ARC-AGI-3 that doesn’t reproduce on a private benchmark opus-5-arc-agi-3.

The caution

Single-benchmark rankings invite gaming and hype; the open-source-llms-2026 source explicitly argues practical fit (task performance, license, hardware, cost, speed) beats leaderboard rank. Treat every ranking as a dated snapshot, and prefer composite/multi-axis views.

llm-leaderboard-stats · llm-provider · llm-api-pricing · open-weight-models · claude-opus-4-8 · arc-agi · arc-prize · opus-5-arc-agi-3