@Rafa_Schwinger - Anthropic Fable's Lead Comes

Open original

@Rafa_Schwinger on X The post argues that Anthropic's Claude Mythos and Fable models owe their edge to verifiable reinforcement learning signal rather than a secret architecture. The author, writing under personal inference, says the lead is measured in months because competitors can replicate the recipe.

  • Capability is modeled as a strong base foundation multiplied by gradeable signal, meaning tasks whose outcomes can be checked cheaply, like code that compiles or passes tests.
  • Long-horizon endurance depends on keeping per-step errors low and managing context tightly; METR puts Mythos's 50% success horizon at 16+ hours, though measurements past that are unreliable.
  • The same verifiability that makes the model strong at coding and security also makes it effective at finding vulnerabilities, with partners reporting roughly 1,500 independently confirmed findings in the first month.

The core argument is that the edge in frontier models now comes from training signal, not architecture. The author's relation is capability ≈ (base foundation) × (gradeable signal extracted on top). Verifiable rewards, meaning tasks where a cheap, reliable check can tell right from wrong, are the scarce input. Raw compute and text are no longer the bottleneck. The author leans on a May 2025 conversation in which Anthropic researchers said reinforcement learning "finally worked" and that verifiability decides where it works. That's more than a year old and predates Mythos, so the author treats it as precedent, not proof of Mythos's pipeline. The piece also cautions that the Fable product is a routed system: sensitive requests in cybersecurity, biology, chemistry, or distillation fall back to Claude Opus 4.8, while Mythos is described as one integrated network. Some of the apparent "panel of models" behavior is just product plumbing, though the author suspects some of the internal paneling may have been folded back into training. The recipe is laid out in layers. The base depends on dense, clean pretraining. The author cites Allen-Zhu and Li, who found that a 1:7 ratio of useful to junk tokens can cut storage efficiency by up to twentyfold, and that a simple provenance tag recovers most of that loss. Rephrasing noisy web text and mixing in about 30% high-quality synthetic data also show large gains, though these are results from other researchers and set the technique class rather than Anthropic's exact recipe. Above that sits reinforcement learning against verifiable rewards, usually a GRPO-style method. The hard part is the reward: it has to track the real task and resist gaming. The author points to Anthropic's dedicated Environment Scaling team as evidence the work is treated as a standing function. One detail stands out. The researchers said they had spent roughly a million dollars on RL against hundreds of millions on pretraining, keeping RL small until they trusted the method, with the implication that spending would grow. Coding gets special attention because it is long-horizon and cheaply checkable: code either compiles and passes tests or it doesn't, which makes it the most valuable environment to build. That's why agentic progress tends to show up in code first. The author also notes that some agentic training data comes from a computer-use patent that arrived via the Adept acquisition, describing trajectories gathered from intercepted human interface actions and on-policy feedback. The disclaimer is worth taking seriously: this is personal inference from public material, and several of the numbers are analogies from other labs' work rather than Anthropic disclosures.

Ask about this article