More on the topic…
Redwood Research proposes a concrete way to measure how much unmonitored reasoning AI models can do behind the scenes. The core problem: chain-of-thought (CoT) reasoning—where models show their work in text—is currently how we oversee what these systems are thinking. But some architectural changes could let models do reasoning in hidden states instead of readable text, making them harder to monitor. The researchers build on prior work by Brown-Cohen et al. that defines "opaque serial depth" as the longest computation path that avoids interpretable checkpoints. Their contribution is pinning down what counts as an "interpretable checkpoint" by introducing the concept of natural-language-rooted (NL-rooted) nodes—places in the model's computation where it outputs human-readable text that was initialized from pre-training.
For a node to be NL-rooted, three conditions must hold: the model was pre-trained to produce natural language at that point (including code, JSON, images, or other human-understandable formats); the outputs stay within what pre-training initialized them to do (no weird abstract reasoning or token superposition tricks); and the model isn't modified to reinterpret those tokens as something radically different post-hoc. Importantly, normal training methods like prompting, distillation, and reinforcement learning don't disqualify a node—even if they make the reasoning harder to read in practice. This definition lets them define NLS depth: the longest computation path that doesn't pass through an NL-rooted node.
NLS depth has practical advantages over just trying to measure how "monitorable" a model is. It's quantifiable before you even train a model—just looking at the architecture and training recipe tells you the answer. This makes it useful for deciding what to build in the first place. The metric works well at distinguishing current language models (which have limited NLS depth even at scale) from genuinely concerning architectures like full-bandwidth transformers that can recur on opaque state indefinitely. The authors aren't claiming this replaces other monitoring approaches, but they argue it's a valuable complement that captures something real about which models can hide their reasoning.
Questions about this article
No questions yet.