More on the topic…
Sean Goedecke explains how to build practical systems around "System One" models—LLMs constrained to output only multiple-choice answers. Unlike traditional language models, these are consistently fast because they batch single-token decisions. You don't need a specialized model to get this behavior; you can convert any LLM into a System One classifier by controlling logits and prefilling prompts. He demonstrates this with a 150-line Python implementation and shows two concrete applications: playing Doom and navigating Wikipedia (Wikiracing).
The Doom demo reveals the first key technique: tiered goals. Simply feeding the model raw game state as choices doesn't work—the model just holds down the shoot button. The fix is periodically asking the model to pick a short-term goal (kill enemies, collect armor) and including that goal in every subsequent decision loop. This mimics how reasoning works in traditional LLMs by allocating more compute to the same problem. The approach scales to real systems with multiple goal-setting loops running at different intervals, from ten-second strategic decisions down to 100ms tactical responses.
The Wikiracing task—navigating from "baseball" to "sun" on Wikipedia—exposes a different problem: scale. With over a thousand links on a single page and a hard ceiling around 100-255 choices before quality degrades, simple ranking doesn't work. Goedecke found that tournament sampling solved this: feed a hundred links at a time, pick winners, then run a second pass. This works because LLMs judge relative quality better than absolute confidence scores. He's optimistic these models will replace tool calls for real-time systems, and expects major labs to release choice-only versions of their fast models soon.
Questions about this article
No questions yet.