This article reruns a 2023 benchmark with the latest LLMs, comparing direct SQL generation against querying through a structured dbt Semantic Layer. It finds that while text-to-SQL accuracy has jumped, a modeled Semantic Layer still delivers near-perfect, deterministic results for covered queries, making it ideal for complex or critical use cases.

• Top 2026 LLMs (Opus 4.6, Sonnet 4.6, GPT-5.3 Codex, GPT-5.2) now hit 80%+ accuracy on raw text-to-SQL, up sharply from the GPT-4 era's ~60%
• A fully modeled dbt Semantic Layer still reaches near-100% accuracy on in-scope queries, versus ~95% for a minimal semantic layer and ~60-80% for plain text-to-SQL
• Data modeling matters more than model choice: basic modeling boosted accuracy 15-20 points, while swapping between top-tier LLMs or reasoning settings only shifted results by single digits
• Semantic layers pay off for complex, messy, enterprise-scale data, while raw LLM text-to-SQL remains a fine low-setup option for one-off or small-dataset analysis