Click any tag below to further narrow down your results
Links
The article introduces Inkling-Small, a Mixture-of-Experts transformer with 276 B total and 12 B active parameters that matches full Inkling performance at one-quarter the size. It handles text, images, and audio with a 1 M-token context window and lets users trade compute for thinking effort. The open-weight model outperforms peers on tool use, coding, reasoning, and multimodal benchmarks and is available for fine-tuning and chat.
- Inkling-Small matches or beats full Inkling performance (e.g., 31.6% vs 29.7% on Humanity's Last Exam) while using only 12B active params out of 276B total—roughly a quarter the compute cost.
- It's natively multimodal (text, images via 40×40 patches, audio via dMel spectrograms) with a 1M-token context window and adjustable "thinking effort" for cost/performance tradeoffs.
- Safety numbers slightly lag full Inkling (71.6% vs 78.0% harmful-query refusal on FORTRESS) despite reusing the same safety recipe.
- Open-weight and available now for fine-tuning on Tinker and chat testing in the Tinker Playground.
The author reruns security vulnerability triage experiments across 26 combinations of Claude and GPT-5 models with varying reasoning effort and context sizes. A four-model “council” achieved 86.2% unanimous votes, and GPT-5.4 at medium/high effort led overall performance, though full-chain solutions remained rare. The study also found higher reasoning sometimes backfires and function-level inputs outperformed whole-file analysis.
- More reasoning effort doesn't reliably mean better vulnerability detection—GPT-5.5-high/xhigh underperformed its own medium setting, and low-effort modes always trailed.
- A 4-model voting council hit 86.2% unanimous decisions and cut no-majority cases to 2.8%, far outperforming any single model.
- Feeding models function-level code instead of whole files dramatically boosted results.
- Even the best setups rarely solved the full vulnerability chain (just 1.9% full solves overall), showing partial pattern-matching rather than genuine deep understanding.
Whichllm is a single-command CLI that detects your GPU/CPU/RAM, pulls live benchmarks from HuggingFace, and ranks the best fitting local LLMs by real performance metrics. It also lets you simulate different GPUs, generate Python snippets, run chats, output JSON, and plan hardware upgrades.
- One command (`uvx whichllm@latest`) detects your GPU/CPU/RAM and ranks local LLMs by live, aggregated benchmarks (LiveBench, Artificial Analysis, Aider, Chatbot Arena, Open LLM Leaderboard) instead of just parameter count.
- Scores are confidence-weighted by data freshness/source (direct vs. self-reported vs. interpolated) and adjusted for runtime fit—VRAM overhead, quantization discounts, and MoE active-vs-total parameter splits.
- It can simulate hypothetical GPUs, reverse-plan what hardware you'd need for a specific model, and output JSON for scripting or piping into jq.
- Example rankings: RTX 4090/3090 tops out with Qwen3.6-27B Q5_K_M (score 92.8, ~27 t/s), 8GB VRAM cards get Qwen3-14B Q3_K_M (score 71.0, ~22 t/s), and CPU-only setups land a 20B MoE model at ~6 t/s (score 45.2).
Ponytail is an always-on ruleset and plugin for AI coding agents (Claude, Codex, Gemini, Copilot, etc.) that enforces a step-by-step “ladder” to include only necessary code. Benchmarks show 80–94% less code, 3–6× faster responses, and 42–75% lower cost by preferring built-ins and one-liner solutions before adding dependencies.
- Ponytail cuts AI-generated code by 80–94% while running 3–6× faster and costing 42–75% less, per Claude API benchmarks across Haiku, Sonnet, and Opus.
- It enforces a strict ladder—skip via YAGNI, use stdlib/native features, add dependencies only if essential, then compress to one-liners—without trimming validation, error handling, security, or accessibility.
- It installs as a plugin (e.g. via Claude Code or Codex commands) and works across Copilot CLI, OpenCode, Gemini/Antigravity, Cursor, Windsurf, Cline, and other agent hosts by copying matching rule files into each tool's config.
- Modes range from lite to full to ultra, adjustable via slash commands or an environment variable, giving control over how aggressively code gets minimized.
The team built a system that quantifies how much engineering work Devin delivers vs. what you pay, then backs that claim with up to $10 million per customer. They validated the methodology with independent data and benchmarks to prove Devin consistently delivers more output than its cost.
- Cognition is offering up to $10 million per customer as a refund/credit if Devin's engineering output doesn't exceed what the customer pays for it.
- The guarantee is backed by an "Engineering Velocity Score" tracking code changes, tests, deployments, and milestones, tied directly to invoices.
- Independent auditors get real-time read-only access to logs to verify data integrity and publish monthly benchmark comparisons against in-house teams, consultancies, and past Devin performance.
- Customers get a portal to track their own Velocity Score in real time, triggering automatic credits if performance falls 10% below the agreed threshold.
This article reruns a 2023 benchmark with the latest LLMs, comparing direct SQL generation against querying through a structured dbt Semantic Layer. It finds that while text-to-SQL accuracy has jumped, a modeled Semantic Layer still delivers near-perfect, deterministic results for covered queries, making it ideal for complex or critical use cases.
- Top 2026 LLMs (Opus 4.6, Sonnet 4.6, GPT-5.3 Codex, GPT-5.2) now hit 80%+ accuracy on raw text-to-SQL, up sharply from the GPT-4 era's ~60%
- A fully modeled dbt Semantic Layer still reaches near-100% accuracy on in-scope queries, versus ~95% for a minimal semantic layer and ~60-80% for plain text-to-SQL
- Data modeling matters more than model choice: basic modeling boosted accuracy 15-20 points, while swapping between top-tier LLMs or reasoning settings only shifted results by single digits
- Semantic layers pay off for complex, messy, enterprise-scale data, while raw LLM text-to-SQL remains a fine low-setup option for one-off or small-dataset analysis