Dario Amodei — Policy on the AI Exponential
AI capabilities are advancing exponentially while policy and legislation lag years behind, creating a dangerous gap. This article argues for binding, FAA-style regulation of frontier models, plus updates to tax, innovation, social power balance, and geopolitical strategies to keep pace.
- AI capabilities went from barely coherent code to handling most software work at top AI firms in four years, tracking scaling laws that predict continued exponential gains
- Frontier models already pose real threats to cybersecurity, finance, critical infrastructure and national security (e.g. the Claude Mythos Preview breach), with bio and autonomy risks likely next
- Voluntary disclosure and optionality-preserving measures (transparency rules, export controls) are no longer sufficient given these demonstrated risks
- Anthropic will back binding federal pre-deployment testing requirements for frontier AI plus job-displacement policy, modeled on an FAA-style certification and audit system, building on early state laws like California's SB 53, New York's RAISE act and Illinois's SB 315
AI is racing ahead while our policy machinery moves at a crawl. In just four years, models went from barely coherent code snippets to handling most of the software work at top AI firms. Empirical “scaling laws” back up the prediction that more compute means exponentially better general intelligence. If that trend holds for another year or two, we’ll face what the author calls “Powerful AI,” a datacenter full of super-smart systems. Meanwhile, Congress can take years to pass even basic rules, leaving regulators perpetually behind.
Early on, safety advocates pushed for measures preserving optionality: transparency rules, chip export controls and data collection on AI’s labor impact. Those steps bought time and built visibility into risks. Now, however, incidents like the Claude Mythos Preview breach show that today’s frontier models already threaten cybersecurity, finance, critical infrastructure and national security. The author warns that biological and autonomy dangers are likely next. With risks undeniable, we must shift from voluntary disclosures to binding rules.
Anthropic plans to back two concrete proposals: a federal framework requiring rigorous pre-deployment testing of frontier models, and a policy package to address AI-driven job displacement. The essay zeroes in on five policy domains in need of overhaul: product-safety rules, macroeconomics and tax, the pace of scientific discovery, the balance between state power and civil society, and geopolitics—all discussed through a US lens but broadly relevant.
On regulation and public safety, the piece argues that AI needs an FAA-style agency. Just as planes undergo technical certification and can be grounded if they fail safety tests, AI models should face mandatory audits and operational reviews. Early wins include SB 53 in California, New York’s RAISE act and Illinois’s SB 315. The Trump administration’s executive order takes steps in the same direction—but we need a permanent, enforcement-driven infrastructure to keep pace with these rapidly evolving systems.