Organizations face three recurring data problems—inconsistent metric definitions, fragmented access controls, and metric changes that don't propagate everywhere. A semantic layer solves this by centralizing metric definitions and governance in one place, so all tools pull the same numbers and changes cascade automatically. It won't fix bad data at the source, but it shrinks the surface area you need to manage and makes self-service analytics actually work.

• The same metric getting defined differently across Tableau, Power BI, and Python isn't a minor annoyance—it's a root cause of bad executive decisions.
• Hiring more BI analysts as gatekeepers just creates ticket queues and bottlenecks; it doesn't fix fragmented governance across tools.
• A semantic layer centralizes metric definitions and access controls so a single change (e.g., redefining ARR) propagates automatically everywhere instead of requiring manual updates across systems.
• Centralizing definitions and logic in one place also makes the data self-documenting, enabling real self-service instead of ticket-based requests.

This newsletter covers major developments across data and AI, including DoorDash’s new Write-Ahead Intent Log for scalable CDC, OpenAI’s Kepler AI analyst for massive data insights, and Lyft’s Metric Semantic Layer for unified business metrics. It also reviews ClickHouse ingestion optimizations, DuckDB’s latest release and agent use cases, Databricks’ Lakehouse//RT and LTAP pushes, plus tools for self-healing pipelines and data quality monitoring.

• DoorDash replaced Debezium with a Write-Ahead Intent Log (WAIL) that writes mutation intents to Kafka and the DB, enabling faster recovery and much higher CDC throughput at Cassandra scale
• MotherDuck claims a $2.40/hour DuckDB instance beats a $64/hour Snowflake 2XL cluster by ~5x on ClickBench, with median query latencies of 3ms, positioning DuckDB for fleets of lightweight agent workloads
• Lyft's Metric Semantic Layer assigns dual business/operational ownership to "Golden Metrics" defined in versioned JSON, auto-propagating changes across dashboards, APIs, and AI agents to prevent divergent calculations
• OpenAI's Kepler analyst now operates over 600 PB of data using Codex-based crawlers and AST-normalized LLM grading to infer lineage, freshness, and hidden semantics automatically

Data lakehouses combine low-cost, flexible storage with warehouse-style governance to power enterprise AI. Companies like DocuSign and Lemongrass use them to feed and train AI agents, but impose strict security reviews, access controls and audit trails. Vendors are adding vector indexing, MCP connectivity and semantic layers to ensure agents grasp business context and operate safely.

• Lakehouses now bolt on vector indexing and MCP connectivity so AI agents can directly query and be trained on enterprise data, with Gartner citing 65% client adoption.
• DocuSign restricts agent access to low-risk data (product specs, web content) while locking down customer records, running every dataset through ingestion and egress security reviews.
• Lemongrass is ditching its four-year-old custom-governed AWS S3 setup for a standard lakehouse, drawn by native Claude integration and zero egress fees when data and models share a cloud.
• Autonomous agents pulling data on their own (vs. per-use-case RAG permissions) demand new audit trails, role-based access and semantic controls to avoid runaway costs and compliance risk.

Rill lets you define BI workflows—models, metrics, dashboards and access policies—in YAML and SQL, and runs queries on engines like ClickHouse or DuckDB. It adds conversational and agent-driven interfaces for natural-language querying and automated dashboard generation, with real-time performance, embeddable dashboards, and Git-backed deployments.

• Full BI stack (models, metrics, dashboards, agent config) spins up from three prompts via `rill init`, then `rill start` gives an instant local UI.
• Metrics-as-code approach: SQL models plus YAML-defined dimensions/measures let Rill generate queries on the fly, returning sub-second results even at billions of rows.
• Same semantic layer powers both natural-language conversational BI and traditional SQL-based analyst workflows.
• Deployment is Git/CI-CD native via `rill deploy`, with dashboards embeddable and metrics exposed through a REST API.

Rill’s Metrics SQL lets you define business metrics once and query them using plain SQL across dashboards, notebooks, and AI agents. It compiles metric views into optimized OLAP queries, handling grouping, filters, time functions, and security automatically.

• Define a metric once in a YAML-backed metrics view and query it as plain SQL from dashboards, notebooks, and AI agents alike, instead of redefining it in dbt, Looker, Python, etc.
• A parser/compiler/executor pipeline rewrites simple queries like "SELECT country, revenue FROM revenue_metrics" into full OLAP SQL with correct GROUP BY, joins, and filters automatically.
• Security (row-level and column permissions) and SQL-injection-safe parameter binding are enforced consistently at the executor stage for every consumer.
• Works across multiple backends (ClickHouse, Snowflake, DuckDB, Druid) and can leverage existing database optimizations like materialized views and indexes for performance.

This article reruns a 2023 benchmark with the latest LLMs, comparing direct SQL generation against querying through a structured dbt Semantic Layer. It finds that while text-to-SQL accuracy has jumped, a modeled Semantic Layer still delivers near-perfect, deterministic results for covered queries, making it ideal for complex or critical use cases.

• Top 2026 LLMs (Opus 4.6, Sonnet 4.6, GPT-5.3 Codex, GPT-5.2) now hit 80%+ accuracy on raw text-to-SQL, up sharply from the GPT-4 era's ~60%
• A fully modeled dbt Semantic Layer still reaches near-100% accuracy on in-scope queries, versus ~95% for a minimal semantic layer and ~60-80% for plain text-to-SQL
• Data modeling matters more than model choice: basic modeling boosted accuracy 15-20 points, while swapping between top-tier LLMs or reasoning settings only shifted results by single digits
• Semantic layers pay off for complex, messy, enterprise-scale data, while raw LLM text-to-SQL remains a fine low-setup option for one-off or small-dataset analysis