Navier-Stokes lost in translation: Why Lean verification of AI autoformalisation does not guarantee correct natural language proofs

Open original

This paper argues that verifying AI-generated mathematical proofs through formal languages like Lean doesn't actually validate the original natural language argument, because translating math text faithfully from English to formal logic is computationally harder than solving the Halting problem. The authors demonstrate this with concrete examples, including OpenAI's claimed proof of Navier-Stokes blow-up, showing the formalized version doesn't match what was originally stated.

  • Resolving ambiguities in mathematical natural language to enable semantically faithful translation has infinite complexity in the Solvability Complexity Index hierarchy — harder than any computable problem including the Halting problem.
  • A Lean proof can pass mechanical verification while the formalized statement diverges from the original natural language proof, creating a false sense of security.
  • OpenAI's announced Navier-Stokes blow-up proof is used as a concrete case where the Lean formalization doesn't correspond to the natural language version.

The paper attacks a growing trend: using AI to translate mathematical proofs from natural language into formal languages like Lean, then claiming mechanical verification proves the original argument correct. OpenAI recently announced a proof of blow-up in Navier-Stokes solutions using this method, which prompted this critique. The core problem is that translation between natural language and formal systems requires resolving ambiguities—a task that sounds routine but is mathematically far harder than it appears. The authors prove this using the Solvability Complexity Index hierarchy, showing that faithful semantic translation has SCI = ∞, meaning it's computationally harder than any finite-level problem, including the Halting problem (SCI = 1). In plain terms: no algorithm can guarantee it solves this translation problem, ever. The practical upshot matters more than the theory. A Lean proof that "verifies" successfully tells you nothing about whether the natural language argument it supposedly formalizes is actually sound. The gap between what was written in English and what got encoded in Lean can hide serious errors. The authors demonstrate this with concrete examples of AI mistranslations, including OpenAI's Navier-Stokes claim. They show that when you examine the actual Lean formalization, it doesn't match the natural language proof—different definitions, different assumptions, different logical structure. The formal verification passes, but it's verifying something else entirely. This matters because it undermines a seductive narrative: that AI-generated proofs can be trusted once they pass a formal checker. The paper reveals why that confidence is misplaced. You can't outsource mathematical rigor to a pipeline that translates through an inherently ambiguous bottleneck. The mechanical verification gives you certainty about the formal statement, but zero certainty about whether that formal statement captures what the mathematician actually meant to prove. For fields like AI safety or critical applications relying on these proofs, that's a significant gap.

Ask about this article