ARC-AGI is a benchmark focusing on genuine reasoning ability in AI, challenging models to identify patterns without relying on memorization. With a $1 million prize for achieving an 85% score on its private evaluation set, it’s a significant indicator of AI progress toward general intelligence. The leaderboard shows rising scores, but the x-axis reflects cost rather than time. Higher scores often require more expense, raising questions about whether increased compute leads to actual advancements in intelligence. The article reveals that the leaderboard is misleading. It captures a moment in time, showing costs from when results were achieved rather than reflecting current capabilities. When viewed as a time series, the efficiency of models has improved dramatically. For instance, on the v1_Semi_Private evaluation set, costs dropped from around $200 per task to just $0.34 within a year for scores between 70-80%. The author emphasizes that the initial high costs associated with breakthrough results often lead to subsequent optimizations that significantly reduce pricing. Three primary factors drive this leftward shift in efficiency. First, training models to instinctively handle ARC-like tasks significantly cuts inference costs. Second, evolutionary test-time compute improves performance through generational refinement of models. Third, enhancements in base model pricing and performance contribute to this trend. The article advocates for tracking the Pareto frontier, which illustrates the evolving relationship between score and cost over time, rather than relying solely on snapshot leaderboards that obscure the real progress being made.