This post lists nine key quotes from a San Francisco talk by Demis Hassabis and Sebastian Mallaby covering everything from OpenAI’s 50% bankruptcy risk to the need for new “AlphaFold” moments in drug discovery. They debate AGI probabilities, frontier cyber defense access, global AI optimism, and the economic and philosophical challenges of a post-scarcity future.

• Mallaby puts a 50% chance OpenAI goes bankrupt within 18 months
• Hassabis says six more AlphaFold-level breakthroughs are needed to cut drug-delivery timelines from 10 years to months
• Hassabis calls the idea that p(doom) is zero "crazy," acknowledging real existential risk from AI
• Hassabis argues economists, social scientists, and philosophers must join scientists in shaping decisions about how AI's value gets shared

The article argues that “AGI” has become a fuzzy umbrella term with conflicting definitions, making it useless for tracking AI progress or predicting major shifts. It urges us to drop AGI in favor of clear, concrete milestones—like fully automated AI R&D, human-level adaptability, or self-sufficient AI—to ground discussions and forecasts.

• "AGI" now spans wildly different thresholds, from matching human test performance to running civilization unaided, making claims like "AGI arrived in early 2025" vs "it's a decade off" talk past each other
• Uneven, jumpy progress across tasks means definitional gaps will keep widening rather than converging
• Concrete milestones (fully automated AI R&D, human-level adaptability, self-sufficient AI, machine consciousness) or vivid benchmarks (10x annual energy growth, interstellar probes) are proposed as replacements for fuzzy labels like AGI or superintelligence
• Swapping vague umbrella terms for specific milestones would make safety, regulation, and timeline discussions more reliable

The article discusses the competitive landscape of artificial general intelligence (AGI) development, likening it to an all-pay auction where participants must invest heavily regardless of the outcome. It argues that this model can lead to inefficiencies and raises concerns about resource allocation in the race towards AGI. The implications of such a competitive framework on innovation and ethical considerations are also explored.

• The AGI race functions as an all-pay auction where every competitor pays their bid (massive capex) regardless of whether they win, driving "value dissipation" toward the total prize value
• Microsoft (>$30B/quarter) and Alphabet (~$85B by 2025) exemplify capex levels that only make sense if losing the race means losing everything already invested
• Because AGI has no agreed definition or finish line, bidders tend to overbid, risking a bubble where combined spending outstrips any realistic returns
• Ordinary investors and pension holders bear outsized risk since most bidders will likely lose while only one winner captures the prize