Therefore I am. I Think
Short summary
This paper explores how large language models make decisions during reasoning. It demonstrates that these models often encode their choices before generating text, influencing their subsequent thought processes. The research shows that altering initial decisions can change reasoning outcomes significantly.
- A linear probe can decode whether a model will call a tool from its activations before it generates any reasoning text, sometimes before any tokens at all
- Artificially flipping this early "decision direction" causes the model to switch its tool-use behavior in 7% to 79% of cases depending on model/benchmark
- When steered toward a different decision, the model's subsequent reasoning rationalizes the new choice rather than resisting or correcting it, suggesting the "thinking" is post-hoc justification rather than genuine deliberation
Full summary
The paper investigates how large language reasoning models make decisions during the generation of text. It poses an intriguing question: do these models think before deciding, or do they decide and then think? The authors present evidence suggesting that decisions can be encoded early in the model's processing, influencing the subsequent reasoning. They employ a linear probe to decode tool-calling decisions from the model's activations before it generates any text, achieving a high confidence level in these predictions. In some cases, decision-making occurs even before any reasoning tokens are produced.
The study highlights the impact of decision direction on the reasoning process. When the decision is perturbed, the model shows increased deliberation and can even switch its behavior significantly, with changes ranging from 7% to 79% based on the model and benchmark used. The researchers further analyze the model's behavior and find that when a decision is steered in a different direction, the reasoning process tends to rationalize this change rather than resist it. This indicates that the models may encode action choices ahead of their deliberative processes, which has implications for understanding how these systems operate and how they can be improved.