Pretraining Data Not Verifiability Is Why Llms Are
Short summary
Steven Byrnes argues that LLMs perform well at math not because math is easy to verify, but because mathematical literature is almost entirely correct—giving the model clean training data to learn from. The "verifiability" explanation is circular since LLMs themselves do the verifying.
- The common claim that "math is easy to verify" doesn't explain LLM math ability, because verification still relies on the LLM's judgment, which is what we're trying to explain in the first place.
- Mathematical literature is >99% accurate, so imitative learning on this data naturally produces correct mathematical reasoning without needing heavy reinforcement learning or formal verification systems.
- The theory predicts which domains LLMs should excel at: those with high-quality, accurate training data (like math and coding) versus those with noisier training signals (like subjective judgments in fiction or social domains).
Full summary
Steven Byrnes pushes back against the popular explanation that LLMs excel at math because math is "easy to verify." He points out this reasoning is circular—saying math is easy to verify really just means LLMs are good at judging math arguments, which doesn't explain *why* they're better at math than other domains. The verifier itself is an LLM anyway, whether you're talking about RLVR (reinforcement learning with verifiable rewards on datasets like AIME) or RLAIF (RL from AI feedback on proofs). Even the mechanical checkers like Lean formal verifiers require models that are already excellent at natural language proofs, making it a chicken-and-egg problem.
His actual theory is simpler: LLMs are good at math because the math literature is almost entirely correct. Pick a random sentence from a research math paper and you can be over 99% confident it's true. Since LLMs learn by imitation—their core strength—they naturally absorb correct reasoning patterns when trained on data where nearly everything is accurate. They don't need some special property of verifiability; they just need high-quality pretraining data where the signal-to-noise ratio is exceptional.
Byrnes frames this within his broader argument that LLMs are still primarily powered by imitative learning rather than reinforcement learning, even after RLHF and other fine-tuning. The math advantage comes down to data quality, not verification mechanisms. This same framework should apply to other domains—areas where LLMs struggle probably contain more noise, contradiction, or ambiguity in their training materials, not because those domains are inherently harder to verify.