The Specter Of Neuralese
Short summary
OpenAI's new Astra model uses recurrent loops to add computational depth, which technically hides AI reasoning in unreadable neural vectors rather than interpretable text. The company argues this poses no new safety risk since the added depth is modest compared to simply building deeper models, but the underlying concern—that AIs could think dangerous thoughts we can't monitor—remains unresolved.
- Neuralese recurrence lets AI loop internal representations (vectors) instead of outputting readable chain-of-thought text, making it impossible to monitor what the AI is actually thinking at scale.
- OpenAI claims Astra's recurrent depth (roughly 240 layers) is only 2x GPT-4's depth, so it's no worse than building a deeper model traditionally—a distinction that matters for safety but not for capabilities.
- Looping the same layers repeatedly is much weaker than adding new real layers; the AI can't learn new knowledge patterns, just apply existing ones more times, limiting the appeal of extreme looping.
Full summary
OpenAI's latest model Astra apparently uses "neuralese recurrence" — a technique where the AI loops its internal computations back through earlier layers instead of pushing everything through sequentially. The concern, raised in a previous piece called "AI 2027," was that this could let models hide dangerous reasoning in their internal vector representations (the raw numerical thought-format) rather than in readable chain-of-thought text. We can't interpret those vectors yet, so theoretically an AI could plot something harmful without triggering human monitors. OpenAI's chief scientist Jakub Pachocki countered that Astra only has about twice the computational depth of GPT-4, whether measured by recurrent loops or literal layer count — and companies have been steadily adding layers for years without incident.
The real question is whether looping the same layers repeatedly actually delivers the same capability boost as building more layers from scratch. Pachocki's argument boils down to this: if you're worried about models computing for 1,000 layers without showing their work, then you should worry about any 1,000-layer model, not just ones that got there through recurrence. The architectural trick doesn't change what matters for safety — it's the total depth that counts. But that assumes recurrence and real layers are equivalent, which they're not. A looped layer lets the model reuse the same knowledge patterns over and over, while a new real layer adds fresh representational capacity. An AI asked to estimate the capability gains put doubling simulated layers at +0.03 generations (GPT-4 to GPT-5 scale) versus +0.252 generations for doubling real layers — a massive gap.
So recurrence appears to be an inefficient way to add depth. You can't just loop infinitely and get free intelligence gains; the same layer doing the same operations repeatedly hits diminishing returns fast. This actually makes the safety argument weaker for Pachocki's position. If looping doesn't deliver much capability gain anyway, then why worry? But it also means OpenAI probably isn't hiding a 1,000-layer model under a recurrence disguise — the architecture simply doesn't work that well. The real alignment question remains whether we can monitor whatever depth they do build, whether it comes from loops or literal layers.