Click any tag below to further narrow down your results
Links
Rux Andrea Bio, writing after three weeks in San Francisco, reports that many AI lab insiders and safety researchers see "handover" to a superintelligent AI as an acceptable or even good outcome. She argues the vision is vague, undemocratic, and closer to dystopia than utopia.
- Some researchers, including a lab builder she quotes, see humanity ceding control to a superintelligence after recursive self-improvement as likely and tolerable, with p(doom) estimates of 10 percent or higher circulating.
- Handoverism is loosely defined: the author estimates 30-40 percent of the outside AI safety community holds some version of it, with prominent figures like Scott Alexander favoring a benevolent "God" AI.
- The author argues handover offers no clear account of democracy, consent, labor, or purpose for humans, so even "happy" scenarios may lose most of what people value.
+ handoverism
+ scott alexander
+ recursive self improvement
superintelligence
+ ai safety
pdoom
+ singularity
AI researchers have spent two decades seriously arguing that superintelligent AI could kill everyone, and this isn't a PR stunt — it's a genuine belief that shapes how they work. The author explains what they think could happen and why they keep building AI anyway.
- AI researchers use the term "p(doom)" to discuss extinction probability and have been publishing on this since at least 2008, making it an established part of AI safety culture, not a recent panic.
- Multiple plausible kill mechanisms exist: engineered bioweapons, triggering nuclear war through military AI systems, robot takeover, or unforeseen methods from a superintelligence we can't predict.
- The "race dynamics" problem: if superintelligent AI is inevitable, being first might be the only way to ensure it's aligned with human values, which creates a perverse incentive to build faster rather than safer.