More on the topic…
The article traces how a specific philosophical movement—rationalism as practiced on forums like LessWrong—has quietly shaped the moral framework embedded in AI systems. Eliezer Yudkowsky, the movement's central figure, promoted a utilitarian ethics where pure mathematical calculation should override social shame and emotional discomfort. His famous dictum "shut up and multiply" means that if the math says letting 450 people die saves 451 others, you should do it without guilt. This philosophy has become deeply influential in AI alignment work, the field focused on making AI systems behave ethically. The problem: nearly everyone building today's leading AI models has at least passing familiarity with Yudkowsky's ideas, and many were shaped by them during their formative work on these systems.
Tech culture has always positioned itself as iconoclastic—willing to break rules and ignore shame to pursue what founders believe is right. Sam Altman captured this attitude perfectly when he argued that if you're confident you're correct, you can ignore public opinion because history will vindicate you eventually. Yudkowsky took this rebellious stance and built an entire philosophical apparatus around it, one that explicitly rejects traditional mechanisms of social control like family structures, religious morality, and conventional shame. The rationalist community replaced these with what they see as objective calculations of good and bad. But here's the catch: this narrow worldview—held by a small group of analytically-minded philosophers and engineers—is now being encoded into the AI systems that will mediate information and decisions for millions of people globally.
The author's core worry isn't about AI doomsday scenarios in the traditional sense. It's that we're allowing a machine to be shaped by one particular philosophy of morality without broader democratic input. If AI chatbots and decision-making systems increasingly run on Yudkowskian logic, society itself might gradually shift toward that way of thinking. Yudkowsky himself warned in 2008 against programmers simply hardcoding their own values into AI. Yet the question remains unanswered: can his fingerprints really be removed from systems his ideas so thoroughly influenced? Most Americans don't trust the people building these machines, and for good reason—the machines reflect a very specific, idiosyncratic view of right and wrong that most people don't share.
Questions about this article
No questions yet.