Click any tag below to further narrow down your results
Links
Microsoft released a narrow model that doesn't generate text. It scores a fixed set of answer options and returns a confidence for each, aimed at routing, classification, and verification in production pipelines. It's built on Alibaba's open-weight Qwen3.5-9B, and most of the performance numbers so far come from Microsoft itself.
- The model returns calibrated probabilities over predefined options in a single forward pass, so applications can act on a score instead of parsing free-form output.
- Microsoft says it beat 36 benchmarks covering about 150,000 questions and runs far faster than GPT-6 Sol on P50 latency, but these are vendor-reported figures with no independent reproduction yet.
- Pricing reported via OpenRouter is $0.042 per million input tokens with free output, and weights aren't released, so access is through hosted APIs only.
Baseten's research lab released two open-source decision models, mammoth-1 (27B) and elephant-1 (9B), built to classify and route inputs by returning probability scores instead of generating text. Both are free to try for a week on Baseten's Model APIs.
- mammoth-1 ranks #1 of 208 decision models on the Jev Decision Index, and elephant-1 is the top 9B model there.
- Decision models return a probability over possible answers rather than writing text, which makes them faster and cheaper than generative LLMs for tasks like ticket routing and spam detection.
- Confidence thresholds let you auto-handle high-certainty cases and escalate uncertain ones to a human.
- Both were fine-tuned from Qwen3.5 on 6.21 million decision-making examples, including training to say "I don't know."
The author compares newer decision models against LLM judges for automated evals on a leasing-agent failure task. The main takeaway is that the default 0.50 threshold often leaves performance on the table, and that the scores need tuning and calibration before you trust them.
- The default 0.50 decision threshold can miss nearly all failures; tuning it per model, using F1 with an 80% precision/recall floor, improves results.
- OpenAI Decisions had the best separation (F1 about 0.87) but cost about 4.5 times more than Perplexity Decider (F1 about 0.79) per 100 eval runs.
- Raw model probabilities are often poorly calibrated; Platt scaling helped Perplexity, Liquid d1, and Clef Flash, but made OpenAI's slightly worse.
OpenAI released the Decisions API, a beta tool that evaluates text and images to return typed answers—probabilities, category selections, or scored ratings—10x faster than the Responses API. It's built for content classification, request routing, and work prioritization without needing to generate free-form text.
- Three question types handle different tasks: predicates return probability estimates for yes/no conditions, choices select from fixed categories, and scores rate inputs against ordered severity levels using probability-weighted averages.
- The API costs $0.10 per 1M input tokens with no output charges, supports HIPAA and Zero Data Retention for eligible customers, and currently runs only on gpt-6-luna model via a dedicated endpoint.
- Real use cases include flagging product photos with visible damage (predicate), routing customer complaints to departments (choice), and assessing bug severity for triage (score).
OpenAI released the Decisions API in public beta, which makes classification and decision-making tasks roughly 10 times faster than their Responses API. The tradeoff is higher cost per request and currently no token caching support, which some developers find economically unattractive for certain use cases.
- Decisions API processes text and image inputs to return typed answers (probabilities, choices from sets, rubric scores) about 10x faster than the Responses API, with median response times around 0.3-0.8 seconds on typical connections
- Accuracy is comparable to Luna (the model used in Responses API) but slightly faster and more variable in performance, though the practical difference is negligible
- No token caching is currently available, making the API more expensive per request than alternatives for repeated classification tasks—a limitation developers hope gets addressed soon
PolicyLM is a 1.7-billion-parameter open-source model that flags policy violations in text by scoring messages against custom rules defined at inference time, without needing retraining. It returns violation scores and threshold-based flags for content moderation across 19 languages.
- Custom policies can be written and applied at inference time without model retraining, making it adaptable to different platform rules on the fly.
- The model scores multiple policy categories in a single pass and is designed for low-latency screening of live chat, comments, and user posts.
- It's released under Apache 2.0 and evaluated on multilingual text, but teams must validate score thresholds on their own data and should only use it for individual messages, not images or conversation context.
Musubi Labs released PolicyLM-1.7B, a small open-source language model designed to make moderation decisions on live messages in under 100ms by reading your platform's actual policies instead of relying on fixed taxonomies. It runs on a single GPU or laptop and lets teams update policies instantly without retraining.
- Processes messages in under 100ms (median 35ms on short chat) while scoring against custom policies, making it viable for live chat where fixed classifiers are blind to policy changes and large LLMs are too slow.
- Reads your policy with every message, so policy edits take effect immediately without retraining or annotation work—useful for teams with frequent rule changes or policies that don't fit standard taxonomies.
- Open weights (Apache 2.0) and already proven in production on a platform handling over a million messages daily; includes two threshold presets (precision-focused for rare violations, balanced for higher detection).
Musubi released PolicyLM-1.7B, a lightweight decision model that applies content policies written in plain English to messages in under 50 milliseconds without needing retraining when policies change. The model costs and runs as fast as traditional AI classifiers but offers the flexibility of modern language models to handle complex moderation rules.
- PolicyLM-1.7B can process content policies in plain English and make binary moderation decisions in under 50 milliseconds, matching the speed and cost of existing classifier systems while avoiding the need for retraining when policies shift.
- Decision models output predetermined outcome probabilities rather than text, making them faster and cheaper than large language models while keeping transformer architecture flexibility — a trend that started with TypeSafe AI's Jev in September, followed by OpenAI and Amazon releases.
- The model gives platform teams a scalable way to label and understand content at scale without building custom training pipelines for each policy change, addressing the challenge of moderating exponentially growing content volumes.
Liquid AI released d1 with image support, a decision model designed for yes/no and classification tasks. It matches or beats GPT-6.1 Sol on most benchmarks while costing 19-200x less and running 200-300ms per decision.
- d1 processes images and text to output probabilities for binary, choice, or scoring questions in a single forward pass without token generation
- Benchmarked against GPT-6.1 Sol and Claude Opus 5.5 on six real applications (support ticket filtering, circuit board inspection, etc.), matching or beating GPT-6.1 Sol on four of them
- Costs 19-200x less than comparable frontier models with significantly faster inference speed
Cloudflare open-sourced two decision models called Clef and Clef-flash that classify inputs into typed, structured outputs with probabilities — useful for routing customer support tickets, identifying phishing domains, or triggering automated agent actions. The models outperform Typesafe's Jev system on speed and benchmarks while supporting images and a 64k context window.
- Clef processes decisions 2x faster than general LLMs (2.2s vs 4.7s in Cloudflare's threat intelligence workflow) because it scores schema choices in parallel rather than generating text token-by-token.
- The models beat Jev on most Jev Decision Index benchmarks and Typesafe's own eval suite, with Clef-flash offering exceptional speed-to-quality tradeoffs; median latency is 209ms for Clef and 38.8ms for Clef-flash.
- Both models are open-sourced on Hugging Face under Apache 2.0, hosted on Workers AI at Cloudflare's edge for low network latency, and fully API-compatible with Jev for easy switching.
Strands released Strands Decider 2B, a 2-billion-parameter model designed to quickly pick between predefined options rather than generate arbitrary text. It runs locally in ~115ms and ranks third among 2B-class decision models on accuracy and calibration benchmarks.
- Decision models sacrifice flexibility for speed and reliability: they choose from given options and assign confidence scores, making them 100x faster than LLMs for simple classification tasks but unsuitable for text generation, coding, or reasoning.
- The 2B parameter size hits a sweet spot for local experimentation—small enough to run on consumer hardware, large enough to handle meaningful work like tool selection, routing, and guardrails in agentic workflows.
- Strands open-sourced the full model, training data, and 19 iterations of architecture refinements on GitHub and Hugging Face, explicitly inviting community contributions to the decision model space.
Researchers tested an AI system called Jev on 2,029 real phone calls stripped down to pure call structure—no audio or transcripts—and it learned to predict which calls would result in bookings with 94% accuracy by the end, all for $3 in compute costs.
- Jev made 38,012 turn-level forecasts across the calls at 118ms median latency, processing thousands of calls in seconds without any fine-tuning or examples from the company's data.
- The system achieved 0.78 AUC by the halfway point and reached 94% ranking accuracy near the end, learning to separate bookable from non-bookable calls based solely on call structure, timing, and workflow patterns.
- The experiment revealed the AI could extract predictive signal from call metadata alone—turns, tool calls, workflow stages—demonstrating that meaningful patterns exist independent of what's actually said.
Jev is a new AI model that classifies data and answers specific questions 193x faster and 444x cheaper than LLMs, but it's fundamentally different—it's a modernized BERT-style classifier, not a generative model. The author tested it on Croatian text classification and found it matched a fine-tuned model's accuracy while costing pennies and taking minutes.
- Jev achieves 97.1% accuracy on zero-shot binary classification tasks, essentially matching fine-tuned BERT models without requiring retraining
- It costs around 20 cents for 3,000 classifications and completes in about a minute, making it practical for real-world workloads where LLMs are overkill
- The model appears more resistant to prompt injection attacks because instructions are separated from input data during training, unlike LLMs where everything blends together
- Jev's "can't hallucinate" claim means it can't generate outputs outside its specified structure (type-safe), but it can still produce incorrect answers within that structure
New lightweight AI models like Jev and SemIf outperform expensive general-purpose language models on straightforward classification tasks—cutting costs by 76-209x while improving accuracy. These tools skip the expensive token-generation process and instead directly evaluate a handful of predefined options, suggesting a future where production systems use cheap specialized models rather than frontier AI.
- Jev and SemIf reduced classification costs by 76-209x compared to frontier models while jumping accuracy from 47% to 80-82% on email thread categorization
- These models work by running attention once and evaluating candidate answers directly from logits, skipping the expensive autoregressive token generation that general LLMs use
- The author replaced roughly 25% of AI calls in their own code with these specialized deciders, suggesting if-then logic is a ripe target for optimization—and hinting other programming primitives could follow
TypeSafe AI released Jev, a model that takes text input but outputs confidence scores and probability distributions instead of generated text. It's designed for classification tasks like spam detection and ranking, costs $0.042 per million input tokens with free output, and has already spawned open-source recreations and creative experiments.
- Jev returns structured probabilistic outputs (yes/no scores, rankings, category choices) rather than text, making it fast and cheap—you only pay for input tokens, not output.
- The model trades interpretability for simplicity: you get a floating-point score but no explanation of which signals drove the decision, raising serious concerns about hidden bias in high-stakes use cases like hiring.
- The community has already built unconventional projects with it (a chat model, a game player, a left-pad implementation) and created open-weight alternatives like Kev within days of release.
Sean Goedecke explains how to build real-time AI systems using "System One" models—fast classifiers that pick from multiple-choice options instead of generating arbitrary text. He shares two practical techniques: tiered goals (breaking decisions into hierarchical loops) and tournament sampling (narrowing large option sets through multiple passes).
- You can convert any LLM into a fast classifier by batching single-token outputs with structured choices, getting predictable inference timing without retraining the model
- Tiered goal hierarchies (strategic → tactical → specific targets) let System One models handle complex tasks like game-playing by providing short-term context, similar to how LLM reasoning chains work
- Tournament sampling outperforms absolute scoring for large option sets—the Wikiracing demo found the optimal 3-link Wikipedia path by sampling 100 links at a time rather than trying to rank all 1,000+ options at once
Using LLMs directly as classifiers is frustrating because they can't be calibrated, don't reliably use all available data, and are hard to improve systematically. Instead, treat LLM outputs as features in a logistic regression or other ML model, which solves these problems while keeping the LLM's signal.
- Direct LLM classification fails on three critical fronts: you can't calibrate confidence scores or adjust precision-recall tradeoffs, the model may ignore structured data or context you provide, and you have no visibility into what the LLM actually used to make its decision.
- Wrapping an LLM verdict in logistic regression automatically fixes calibration (empirically matching true probabilities), lets you add other features, and gives interpretability about how much the LLM contributes to the final decision.
- On an irony detection task, logistic regression on the raw LLM verdict improved Brier score from 0.259 to 0.175 just by calibrating; adding more LLM-extracted features (token probabilities, multiple reasoning passes) provides a clear path to further improvement that prompt-tweaking doesn't offer.