The top ranked decision models in the world: mammoth-1 and elephant-1
Short summary
Baseten's research lab released two open-source decision models, mammoth-1 (27B) and elephant-1 (9B), built to classify and route inputs by returning probability scores instead of generating text. Both are free to try for a week on Baseten's Model APIs.
- mammoth-1 ranks #1 of 208 decision models on the Jev Decision Index, and elephant-1 is the top 9B model there.
- Decision models return a probability over possible answers rather than writing text, which makes them faster and cheaper than generative LLMs for tasks like ticket routing and spam detection.
- Confidence thresholds let you auto-handle high-certainty cases and escalate uncertain ones to a human.
- Both were fine-tuned from Qwen3.5 on 6.21 million decision-making examples, including training to say "I don't know."
Full summary
Baseten's research lab has released two open-source "decision models," mammoth-1 (27B parameters) and elephant-1 (9B), both post-trained from Qwen3.5 base models. Unlike generative LLMs, these models don't write text. They output a probability distribution over a fixed set of answers, so you can ask them multiple-choice questions (which team owns this ticket?), scale ratings (how urgent is this, low or high?), or yes/no questions (is this a phishing email?). The pitch is that for classification-style work like ticket routing, spam filtering, or jailbreak detection, generating a full paragraph of reasoning is wasted compute. Skipping generation makes these models cheaper and faster, which the authors claim can be up to 10x quicker than a reasoning model, and the probability output gives you a built-in confidence score you can threshold on, for instance auto-routing a ticket at 90% confidence and escalating anything below that.
On benchmarks, the article says mammoth-1 ranks #1 out of 208 models on the Jev Decision Index 0.4, while elephant-1 is tied for #8 overall and is the top 9B decision model. On the separate Artificial Analysis Decision Index, elephant-1 ties for #1 with the previous best, d1-ultra. The rule of thumb offered is that mammoth-1 suits tasks needing broader general knowledge, and elephant-1 suits quick reasoning at lower cost. Both were trained on 6.21 million decision examples drawn from public datasets and synthetic data covering five domains: knowledge and reasoning, language, tools and automation, retrieval and classification, and arts and human taste. The training data included examples teaching the model to say "I don't know" when it is uncertain, which matters if you plan to rely on its confidence scores. Full details are in a technical report from Base Labs.
Practically, both models can be tried free for a week through Baseten Model APIs, with zero data retention on by default, and the weights are on Hugging Face. The article's advice on evaluating them is sensible and boring in a useful way: test accuracy on hard cases rather than easy ones, check whether the confidence scores actually track correctness, and measure latency and cost per decision at your real traffic volume, including retries. Note that every performance claim here comes from Baseten's own framing and the indices they cite, so running your own eval on your own data is the step that will tell you whether either model fits your stack.