Click any tag below to further narrow down your results
Links
The author, a former theoretical physicist, argues that superintelligence won't cause the explosive economic change rationalists predict, because physical, regulatory, and informational bottlenecks cap how much faster the overall world can move. He uses Amdahl's Law and a widget-factory example to show that speeding up one step rarely speeds up the whole process by much.
- Amdahl's Law: if a process has several sequential bottlenecks, infinitely speeding up one still leaves a constant-factor overall speedup, so exponential gains require repeatedly removing the current bottleneck.
- In the factory example, even with omniscient AI agents working 24/7, a month-long fix only drops to about a week because of shipping times, lab processes, and installing physical hardware.
- Some bits of information are very expensive to acquire (like long-term side-effect data), and regulation and political will are slow speed limits that intelligence doesn't remove.
Three AI researchers from frontier labs discuss why transformative AI might fail to emerge despite current progress—exploring technical bottlenecks like the sim-to-real gap, the possibility that current methods hit diminishing returns, and whether scaling alone can discover the next paradigm shift needed for AGI.
- Current AI progress follows repeating cycles where new models seem revolutionary but reveal limitations within months, potentially continuing indefinitely without reaching true generalization or self-improvement loops.
- The transformer + RL paradigm may require fundamental discontinuities to advance further, similar to how scaling laws in deep learning repeatedly hit walls that required new innovations (pre-training, then RL) to overcome—and future breakthroughs might not be discoverable by scaling current methods alone.
- Even if AI systems become better than humans at research, the jump to explosive recursive self-improvement isn't guaranteed; there could be persistent gaps between what works in simulation and real-world deployment that prevent the takeoff scenario.