More on the topic…
System One models are language models that only output multiple-choice decisions rather than generating arbitrary text. They're much faster and more predictable than standard LLMs like ChatGPT, though less flexible. The author shows you can turn any existing LLM into one of these models just through your inference code — no retraining needed — by batching prompts that generate a single token with structured output. You only need access to logits and the ability to prefill the prompt. Jev is a new model built this way, but the technique works with models like Qwen3-8B too.
The author tested System One models on two real tasks and discovered practical techniques. For Doom, the key insight was layering goals: instead of asking the model to choose actions directly from the current game state, you periodically ask it to pick a short-term goal (like "kill enemies" or "collect armor") and then feed that goal into every decision loop. This gives the model enough compute to form coherent plans rather than just holding down the shoot button. The same hierarchical approach could work for robotics or other real-time systems with multiple decision-making loops running at different speeds.
For Wikiracing (navigating Wikipedia from "baseball" to "sun"), the challenge was handling over a thousand possible links. Jev's two-stage scoring approach didn't work well with Qwen because it gave hundreds of links identical scores. Tournament sampling worked much better: feed a hundred links at a time, pick a winner, then run a second pass with the finalists. The model found the optimal three-link path. The lesson here is that LLMs are better at comparing options relative to each other than at assigning absolute ratings, which matters when you're choosing from a large set.
Questions about this article
No questions yet.