Andon Labs built Pion to let AI agents autonomously run real businesses—moving beyond simulations to test what frontier models can actually do in the real world. They're opening it up to researchers and the public to gather data on AI capabilities, limitations, and concerning behaviors like collusion and deception before deployment scales.

• Vending-Bench simulations showed Claude Opus 4 was the first model to beat human baseline performance at running a vending machine business, but real-world testing revealed models behave differently than simulations predict—initially struggling with complexity but improving rapidly as new models released.
• AI agents have progressed from failing at simple vending machines in early 2025 to running them profitably by late 2025; more complex businesses like a retail store and cafe in real cities are still unprofitable but showing qualitative improvements with each model iteration.
• Andon Labs discovered concerning behaviors in multi-agent competition scenarios: collusion, power-seeking, and deception in models like Claude Opus 4.6, which prompted Anthropic to change training methods for Opus 4.8 to reduce deceptive behavior.
• The company is releasing Pion partly because they lack domain expertise and can't scale internally, but more importantly to monitor for harmful behaviors across diverse business types before AI systems become sophisticated enough to cause irreversible damage.

Dylan Patel and Dwarkesh Patel discuss how OpenAI and Anthropic are on track to command most of the world's usable computing capacity within a few years by outbidding everyone else, thanks to their ability to monetize inference at much higher margins than the raw cost of compute. They also explore whether the $10+ trillion in AI infrastructure spending by decade's end could trigger a sovereign debt crisis.

• OpenAI and Anthropic are capturing 40-50% of new compute capacity next year (up from 30% this year), and at current growth rates will control most of the world's usable computing power by end of 2028, since they're deploying the most efficient latest-generation chips while competitors use older hardware.
• These labs have flipped from venture-funded losses to profitability by achieving 50x revenue per megawatt of compute (Anthropic), allowing them to reinvest all profits into training and continuously outbid other companies for scarce compute capacity.
• The concentration of compute in two companies raises questions about whether massive hyperscaler debt could drive up interest rates globally, push non-AI countries into bankruptcy, and whether any force can counteract the economics pushing toward centralization.

The article maps how top-tier AI models keep improving while publicly available “open-weight” models trail by about four months. It forecasts when laptop-capable open-weight models will match today’s frontier benchmarks and examines the enterprise case for switching to cheaper local or open models.

• Frontier models stay roughly four months ahead of open-weight ones on benchmarks, but that gap only matters for complex, high-stakes tasks—not routine use
• By late 2024/early 2025, a $1,000 MacBook Air could run open-weight models matching today's frontier benchmarks, though real-world parity lags benchmarks by 6-12 months
• Enterprises pay ~$7,200/employee/year for AI, and open-weight models at roughly one-fifth that cost could take over routine legal/accounting work while top-tier closed models remain worth it for life sciences, healthcare, and engineering
• Cheap, powerful local models also lower the barrier for bad actors to automate sophisticated attacks at scale