Xinming Tu | Claude was trained on protein sequences at scale

Open original

The authors test whether frontier LLMs can predict the next amino acid in obscure bacterial proteins that almost no one writes about, comparing real sequences against shuffled ones to isolate genuine sequence knowledge. The latest Claude models (Opus 5.5 and Fable 5.2) do this well, while GPT, Gemini, and open-weight models stay near zero.

  • On 668 little-studied proteins, Fable 5.2 predicts the next residue correctly 54% of the time on real sequences versus 9% on shuffled ones; Opus 5.5 behaves like a dedicated protein language model.
  • The gap grows with family size, which fits a model that learned from sequence data rather than from papers, since the effect holds even in families with no publications.
  • GPT, Gemini, and seven open-weight models show essentially no gap, and DNA prediction is weak, so the authors conclude the Claude models learned proteins specifically, not genomes.

The core finding is that the latest Claude models, specifically Fable 5.2 and Opus 5.5, appear to have learned protein sequences at scale, while GPT, Gemini and seven open-weight models show essentially no such ability. The test is clever in its setup: give a model the start of a little-studied bacterial or archaeal protein and ask for the next amino acids, then repeat on a shuffled copy of the same protein. Shuffling keeps the letters but destroys their order, so the gap between real and shuffled accuracy isolates what the model actually knows about sequence structure. On 668 proteins from 167 Pfam families of unknown function, Fable 5.2 gets the next residue right 54% of the time on real sequences and 9% on shuffled ones. Opus 5.5 is level with ProtGPT3-10B, the largest open protein language model the authors ran. Both results hold in families with no papers at all, which rules out the simplest explanation that the models are just recalling literature. The authors also check whether the effect behaves like a protein language model should. It does: accuracy climbs with family size, with Fable 5.2 rising fastest and leveling off around 300 proteins per family, and the earlier Claude models rising at a lower level. Reasoning doesn't explain it either, since most answers involve no thinking at all and the non-thinking answers are just as accurate. Famous proteins like GFP are a different matter, and many models recite them nearly letter for letter, so the authors deliberately used obscure families to shut that route out. They also argue that some earlier results on well-known proteins, like SOD1 variant ranking, may have leaned on text about those proteins rather than sequence knowledge. The DNA controls are a useful contrast: Opus 5.5 predicts human and mouse coding sequence but not the wider genome, so what it learned looks specific to proteins rather than to biology in general. The practical caveats are worth keeping in mind. These are single-company blog-post results on models whose training data isn't public, the test uses a small set of bacterial and archaeal proteins, and the comparisons are against models run at their lowest thinking setting. The claim that Claude saw protein sequences at scale is well supported by the shuffle control and the family-size trend, but the exact amount and source of that data remains unknown. The authors have released code, test sets and outputs, so the numbers can be checked directly rather than taken on trust.

Ask about this article