1 link tagged with all of: ai-safety + transformer-architecture + rlhf
Click any tag below to further narrow down your results
Links
Engineers from Anthropic break down Claude’s design, covering its transformer-based architecture, data curation methods, and reinforcement learning from human feedback. They also dive into safety measures and guardrails built to curb harmful or biased outputs.
- Claude's "constitutional AI" approach uses one model instance to critique and another to rewrite responses against a fixed rule set, cutting harmful outputs by ~50% versus standard RLHF alone
- Claude 2 (52B parameters) edges out GPT-4 on ARC-S science reasoning (79% vs 78%) while roughly matching peers on HumanEval code generation (~65%)
- Critique and rewrite stages run on physically separate clusters, meaning a single compromised node can't both judge and produce outputs
- Training data is kept in-house rather than outsourced to contractors, reducing leak risk