Two techniques for working with System One models
Short summary
Sean Goedecke explains how to build real-time AI systems using "System One" models—fast classifiers that pick from multiple-choice options instead of generating arbitrary text. He shares two practical techniques: tiered goals (breaking decisions into hierarchical loops) and tournament sampling (narrowing large option sets through multiple passes).
- You can convert any LLM into a fast classifier by batching single-token outputs with structured choices, getting predictable inference timing without retraining the model
- Tiered goal hierarchies (strategic → tactical → specific targets) let System One models handle complex tasks like game-playing by providing short-term context, similar to how LLM reasoning chains work
- Tournament sampling outperforms absolute scoring for large option sets—the Wikiracing demo found the optimal 3-link Wikipedia path by sampling 100 links at a time rather than trying to rank all 1,000+ options at once
Full summary
System One models are language models designed to output only multiple-choice answers rather than free-form text, making them consistently fast but less flexible than ChatGPT. You don't need a specialized model like Jev to get these benefits—you can convert any existing LLM into a System One classifier by batching prompts that generate single tokens and using structured output. The author built a working version in about 150 lines of Python and tested it on two real tasks to figure out how to actually program with these constrained models.
When the author implemented Qwen3-8B playing Doom, a naive approach of just presenting game inputs as choices failed completely—the model would hold down the shoot button indefinitely because a single 200ms forward pass didn't provide enough compute to figure out what it should actually do. The fix was adding a tiered goal system: periodically ask the model to pick a short-term objective like "collect armor" or "kill enemies," then include that goal in every subsequent decision loop. This mirrors how game AI traditionally works, with nested loops running at different timescales. The author suggests a four-level hierarchy could work: strategic goals every ten seconds, tactical subgoals every five seconds, specific targets every second, and raw input decisions every 100ms.
For Wikiracing—navigating from Wikipedia's baseball page to the sun page—the bottleneck was scale: baseball has over a thousand internal links, but the model struggles to pick well among more than a hundred choices. The author found that Jev's two-stage approach (score independently, then choose) didn't work because Qwen3-8B couldn't produce meaningful confidence scores. Tournament sampling worked instead: feed a hundred links at a time into choices, then run a second pass on the winners. This plays to LLMs' actual strength—making relative judgments between options rather than assigning absolute scores. The model found the optimal three-link path: baseball → scientific american → amateur astronomy → sun.