AI Scaling vs Human Scaling — Toby Ord
Short summary
Toby Ord examines whether AI systems improve as efficiently as humans when given more resources — specifically, whether 10x more tokens produces the same performance gain as 10x more time does for humans. Recent data suggests AI has scaled worse than humans historically, but newer models are closing the gap.
- Early data from METR showed humans gained ~4x score per resource doubling while AI gained less, implying AI would need 10,000x more tokens to match a human's 10x time improvement — but this used weak models without reasoning capabilities.
- UK AISI's July 2026 chart tracking how long AI can work on tasks (measured in human time-equivalent) shows all recent models scale sublinearly with tokens (γ < 1), meaning 10x tokens yields less than 10x improvement, but this ratio has improved dramatically from 10,000x to just 20x in under a year.
- The scaling relationship appears to be improving over time, with γ rising from near 0.1 to possibly around 0.67, raising the possibility that newer models could eventually match or exceed human scaling efficiency (γ = 1).
- This matters for recursive self-improvement models: if AI can eventually scale as well as or better than humans with additional resources, it changes whether an intelligence explosion becomes plausible.
Full summary
Toby Ord investigates whether AI models improve from extra computational tokens at the same rate humans improve from extra time on tasks. The comparison matters because it affects predictions about AI capabilities and whether systems could self-improve fast enough to trigger runaway intelligence growth. If AIs need 100x more tokens to match the performance gain a human gets from 10x more time, the analogy breaks down entirely. The core challenge: good data is scarce because it requires measuring both AI and human performance on identical tasks across different resource budgets.
A METR study showed humans scaling about 4x better per resource doubling than early AI models, suggesting AIs would need roughly 10,000x more tokens to match a 10x human time improvement. But this data has real limitations—it used older, non-reasoning models and mixed different scaling types (chain-of-thought versus trying multiple times). More recent AISI data from hacking tasks, plotted on a log-log graph, reveals all tested models scale sublinearly with tokens. Each model requires a different multiplier: older systems needed roughly 10,000x token scaling for 10x time-horizon improvement, but by May 2026, newer models needed only 20x. The improvement trajectory shows γ (the scaling exponent) rising from near zero toward roughly 2/3, though it's unclear if this trend continues or has plateaued.
The data suggests AI systems currently scale worse with inference compute than humans scale with time, but the gap is closing fast. Whether models will eventually match human scaling efficiency or even exceed it remains open. Ord emphasizes the uncertainty: the experiments involve single task suites, extreme chain-of-thought lengths requiring multiple compaction rounds, and substantial measurement error. More work tracking this across diverse benchmarks and newer models is needed before drawing firm conclusions about AI's trajectory.