More on the topic…
OpenAI's GPT-6 Astra just posted remarkable results on ARC-AGI-3, a benchmark designed to measure how well AI systems can learn and solve novel abstract puzzles. The key numbers: with the standard setup, Astra hit 62.7% accuracy for $26K in computational cost, but with an advanced harness that lets it preserve reasoning state between attempts, it jumped to 99.9% for $19K. More striking than the scores themselves—Astra beat human performance on action efficiency, using fewer moves than the median human on 96% of levels and averaging 51.7% fewer actions per puzzle. For context, humans in the testing cost roughly $12.78 per attempted game when you factor in their time, though the actual brain energy is closer to 0.067 cents per game.
What separates Astra's performance from brute-force approaches is how it actually solves these puzzles. The model develops compact symbolic representations of game mechanics, essentially creating its own algebraic shorthand to track state and plan moves. In one example, it condensed a complex puzzle into notation like "L8: hub q2 (8↓). Lengths: 14=1..." to track objects, coordinates, and interaction rules. It also builds ordered sequences like "extend8 to3; retract10 to2; shorten8 to1" to map multi-step solutions. This isn't random trial-and-error—once Astra grasps the underlying mechanics of an environment, it executes solutions with roughly human-level efficiency, sometimes better.
The benchmark itself matters because it tests four specific agentic capabilities: exploration (actively gathering information), modeling (turning observations into predictive frameworks), goal-setting (identifying targets with sparse feedback), and planning with course correction. ARC-AGI-3 is the third generation of this series, and it deliberately escalates beyond previous versions as AI capabilities improve. The designers calibrated difficulty through testing with actual humans, who can solve 100% of the environments—so there's no ceiling effect hiding what the residual gap between current AI and true AGI actually is.
Questions about this article
No questions yet.