Two techniques for working with System One models

Open original

The author demonstrates two practical techniques for programming with System One models (fast classifiers that pick from multiple choices): tiered goal-setting for sequential decision-making, and tournament sampling for choosing among many options. He shows these approaches working in Doom gameplay and Wikipedia navigation tasks.

  • Tiered goals work better than single-pass decisions: periodic prompts asking the model to choose short-term goals (e.g., "kill enemies" vs "collect armor") make it perform more intelligently than just reacting to immediate game state every 200ms.
  • Tournament sampling beats confidence scoring when picking from many options: feeding 100 links at a time, then narrowing down, found the optimal path in seconds, while trying to score all 1,000+ links failed badly.
  • System One models offer a practical alternative to tool calls for real-time systems where you need predictable latency and don't need full language generation flexibility.

System One models are language models that only output multiple-choice decisions rather than generating arbitrary text. They're much faster and more predictable than standard LLMs like ChatGPT, though less flexible. The author shows you can turn any existing LLM into one of these models just through your inference code — no retraining needed — by batching prompts that generate a single token with structured output. You only need access to logits and the ability to prefill the prompt. Jev is a new model built this way, but the technique works with models like Qwen3-8B too. The author tested System One models on two real tasks and discovered practical techniques. For Doom, the key insight was layering goals: instead of asking the model to choose actions directly from the current game state, you periodically ask it to pick a short-term goal (like "kill enemies" or "collect armor") and then feed that goal into every decision loop. This gives the model enough compute to form coherent plans rather than just holding down the shoot button. The same hierarchical approach could work for robotics or other real-time systems with multiple decision-making loops running at different speeds. For Wikiracing (navigating Wikipedia from "baseball" to "sun"), the challenge was handling over a thousand possible links. Jev's two-stage scoring approach didn't work well with Qwen because it gave hundreds of links identical scores. Tournament sampling worked much better: feed a hundred links at a time, pick a winner, then run a second pass with the finalists. The model found the optimal three-link path. The lesson here is that LLMs are better at comparing options relative to each other than at assigning absolute ratings, which matters when you're choosing from a large set.

Ask about this article