Click any tag below to further narrow down your results
Links
OpenAI has made its Decisions API generally available, a fast, cheap model for picking among options with a confidence score, meant to compete with TypeSafe's Jev. The pitch is that for many workflows, a quick classifier-style model can replace a full LLM.
- OpenAI didn't build a dedicated decision model; it repurposed its ChatGPT Luna model and tuned it for lower latency and cost.
- Speed is the main selling point: response times dropped from 1-3 seconds to roughly 100-180 milliseconds, and Speak cut its action gap from 1.4 seconds to 0.13 seconds.
- Benchmarks don't settle which decision model is best. Jev ranks well on some indices and tops the PacMan-based JevMan test, while OpenAI's model ranks lower, though it may have an edge in image classification.
@julianharris on X
Julian Harris argues that Jev, a startup that popularised a novel way to use LLMs, got copied almost instantly, with Cloudflare's open-source Clef beating it on speed and accuracy within two weeks. The post is a warning that without proprietary data, software offers little competitive moat.
- Clef uses a frozen Qwen model that does a single prefill pass, with a small schema head scoring all answer options in parallel and generating no text.
- The approach reportedly runs about 4x faster than Jev and reaches 2x the accuracy on some benchmarks.
- Clef is open source, with weights published on Hugging Face, which makes the copy easy for anyone to adopt.
@JayaGup10 on X
The post argues that the all-purpose LLM is splitting apart: judgment, ranking, search, and verification are getting priced and built as separate cheap layers, with frontier models reserved for the hardest work. It introduces Jev from TypeSafe as an example of packaging one decision step as its own model call.
- Agents make hundreds or thousands of model calls per task, so small per-step cost and latency differences add up to the whole product's economics.
- A "decoder tax" is paid when a model generates a full sentence just so the app can turn it back into a yes/no, urgent/not-urgent, or which-tool decision.
- Cheap enough judgment lets software check 100% of agent actions and customer interactions instead of sampling 1%, which opens product categories that aren't economical today.