Encoding Your Domain Expert: The Context Layer Behind Spotify's Data Assistant | Spotify Engineering

Open original

Short summary

Spotify built Vedder, an AI data assistant that uses domain-specific “clusters” curated by expert teams to provide reliable SQL-generated insights. Each cluster combines relevant datasets, vetted question–SQL pairs, and docs, with automated health scoring and feedback loops to keep context accurate and scalable.

  • Auto-generating question-SQL examples from query logs mostly failed—only 12.5% passed expert review, proving curated human examples beat scraped ones
  • Vedder's reliability comes from domain-expert-curated "clusters" (tables + vetted SQL examples + docs), not a bigger LLM
  • Health scoring tracks schema drift and broken examples so experts know which clusters need fixing as data evolves
  • Adopted by 2,100+ employees for 13,000+ conversations since August 2025, showing real traction beyond a pilot

Full summary

Spotify ran into a scaling wall when teams kept pinging data experts for insights. Dashboards weren’t enough, and there were over 70,000 datasets in petabytes of data. Simply dumping all schemas into an LLM failed—context windows cap out and raw schemas don’t explain business nuances. To bridge that gap, Spotify built Vedder, a data assistant that picks the right context, writes SQL, runs it, and returns both answers and the queries that generated them. It plugs into Slack, IDEs and a web UI, and more than 2,100 employees have held 13,000+ conversations with it since August 2025. The secret sauce isn’t a bigger model but a curated “cluster” layer owned by domain teams. Each cluster groups relevant warehouse tables with profiling and samples; vetted question-SQL examples written or approved by experts; and business docs that spell out terminology and gotchas. Those curated examples are the few-shot seeds that teach the model how to query and interpret data. An experiment that auto-generated examples from query logs produced a deluge of noisy, misleading pairs—only 12.5% passed expert review. Human judgment remains the gatekeeper of trust. To keep clusters accurate as data evolves, Vedder tracks health scores using signals like schema drift, broken examples and real-world usage. Domain experts see dashboards flagging which clusters need attention. Every interaction with Vedder—questions, generated SQL, user feedback—feeds back into cluster dashboards. That loop lets experts refine context continuously. Spotify’s architecture isn’t tied to its tech stack. Any organization with domain experts and a data warehouse can layer in curated context, so LLMs deliver reliable insights instead of guesses.

Ask about this article