2 links tagged with all of: alignment + claude + anthropic
Click any tag below to further narrow down your results
Links
TheZvi argues that discussions about AI consciousness are plagued by sloppy reasoning and conflicting motivations, while public opinion is gradually shifting toward treating current AI systems as potential moral patients. He pushes back on both dismissive claims that AIs can't be conscious and on framing that treats the question as settled.
- Opinion on AI moral patienthood has shifted noticeably: "probably yes" for current models rose from 15% to 35%, while 80% expect AIs to become moral patients within their lifetimes, suggesting a threshold crossing is expected.
- Arguments against AI consciousness often conflate distinct philosophical questions (sentience, minds, moral status) and rely on weak reasoning like treating intuition as decisive or using analogies without checking structural parallels—Henry Shevlin notes these failures would fail undergraduate essays.
- Anthropic's policy banning sustained abuse of Claude and its meetings with religious leaders on consciousness are sincere attempts to grapple with hard questions, not marketing, because taking the issue seriously predictably invites derision rather than praise.
- The core problem remains unsolved: if AIs become conscious, society faces a genuine dilemma between importing "immigrants from artificial realms" (cheap labor but potential moral patients) and avoiding that risk entirely.
The article discusses the new version of Claude's constitution, which outlines explicit values for AI behavior. It explains how Constitutional AI improves upon traditional human feedback by using AI-generated principles to ensure safer and more transparent model outputs. The principles aim to address ethical concerns while allowing for continuous improvement.
- Constitutional AI replaces most human feedback with AI self-critique against a written constitution, then AI-generated reinforcement learning for harmlessness, cutting the need for people to review toxic content.
- Claude trained this way got better at handling adversarial prompts without becoming less helpful, and the approach makes the values steering the model's outputs more transparent.
- The constitution draws on sources like the UN Declaration of Human Rights and other labs' safety practices, but Anthropic admits it's still skewed toward Western viewpoints and remains a work in progress shaped by trial and error.