1 link tagged with all of: inference-optimization + language-models + system-design + structured-output
Click any tag below to further narrow down your results
Links
System One models are stripped-down LLMs that only output multiple-choice answers, trading flexibility for speed and predictability. The author shows two practical techniques for building real-time systems with them: layered goal-setting for sequential decision-making and tournament sampling for choosing among many options.
- You can convert any LLM into a fast classifier by batching single-token outputs with structured prompts—no model retraining needed, just inference-level changes.
- Tiered goals (asking the model to pick short-term objectives before making immediate decisions) dramatically improves performance in tasks like game-playing by giving the model more compute to reason about strategy.
- Tournament sampling—splitting large choice sets across multiple rounds rather than trying to rank everything at once—works better than absolute scoring because LLMs judge relative quality more reliably than absolute confidence.