Click any tag below to further narrow down your results
Links
This page describes the Artificial Analysis Decision Index, a weighted benchmark that scores small "decision models" on fast classification tasks like routing support tickets and rating urgency. It combines six evaluations, from decision-making and tool calling to hallucination and answer invariance, into one composite score.
- Decision models answer every question in a single call with a probability for each option, which keeps them cheap and fast and makes calibration measurable.
- The index weights AA-Paperclip (document extraction as one decision) at 30%, with five other evaluations at 10–15% each.
- Accuracy counts the top-probability option as the answer, and the page also reports Brier scores, time per task, and cost per 1M tasks.
@OneLegDavePDX on X
Google has released Gemini Decision 1, a model that answers only yes, no, or by picking from a fixed list, producing zero output tokens. The post mocks it as a fast alternative to verbose chatbots, noting Google's own showcase use case is choosing emojis.
- Gemini Decision 1 outputs only yes, no, or a choice from a list, with zero output tokens.
- The post contrasts it with chatbots that return hundreds of words for simple questions like whether a build passed.
- Google's own showcase example is picking emojis, which the author presents as the joke.