5 Jev use cases for analytics, with real queries and datasets | MotherDuck
Short summary
MotherDuck's Jev is a decision model that classifies text by picking from predefined answers, running 30x faster than GPT at 1% of frontier model costs. The article walks through five real use cases—grading AI agent logs, analyzing Hacker News sentiment, screening sales calls, and others—with actual SQL queries and datasets you can attach and rerun.
- Jev classifies 100,000 items in 82 seconds at ~1% the cost of frontier models by constraining outputs to a closed list you define, returning typed columns with confidence scores.
- On Hacker News mentions, 89% of pre-2023 "Claude" and "Anthropic" matches were false positives (Claude Shannon, the anthropic principle), but Jev's name-checking reduced false positives to 0.8% post-2023 when the products actually launched.
- The pattern separates facts from judgment: SQL extracts exact counts from logs or data, Jev only answers subjective questions like "was this turn efficient?" or "is this negative?", keeping costs down while maintaining accuracy.
Full summary
Jev is a decision model that classifies text by picking from a closed set of options you define, then returns a typed column with confidence scores—all fast enough to run on every row in SQL via a `prompt_jev()` function on MotherDuck. The article walks through five real-world use cases with actual queries and datasets you can test yourself. The core appeal: it classifies 100,000 complaints in 82 seconds at roughly 1% of what frontier models cost, making it practical for high-volume labeling tasks.
The first use case grades AI agent logs by scoring whether each turn completed the user's request and whether it wasted tool calls. Instead of running expensive judgments on everything, they score every turn cheaply, then ask follow-up "why" questions only on turns that score below a threshold. The design separates facts from judgment—SQL counts exact metrics like repeated calls, while Jev only answers what requires subjective evaluation. A second example uses Jev for sentiment analysis on Hacker News, where it filters out noise: before 2023, 89% of comments matching "claude" or "anthropic" were about Claude Shannon or the anthropic principle, not the AI labs. After the models shipped, that dropped to 0.8%. Gemini stays messier at 3.6% false positives because the name collides with NASA's program, a network protocol, and a crypto exchange.
The pattern works because Jev handles ambiguity that regex or simple keyword matching can't touch. You feed it context—dates, product names, full comment text—and ask a binary or scored question. The trick is writing clear instructions that list what counts as each answer, then letting the model make the judgment call consistently across millions of rows. The examples show this beats building custom classifiers and costs less than running everything through GPT-4-level models, making it useful anywhere you need to label data at scale without building a full ML pipeline.