Click any tag below to further narrow down your results
Links
DuckDB 2.0 Alpha adds dynamic attach statements that let you pull credentials from environment variables instead of hardcoding them, solving a major friction point when working across multiple data lakes. Early production testing shows less than 1% failure rates, and the release bundles this with Quack and official DuckLake support to make local-first lakehouse workflows actually practical.
- Dynamic attach statements now accept environment variables for credentials (usernames, passwords, account numbers), eliminating the need to hardcode sensitive data or switch to Python for multi-catalog federation
- Production pilot running hourly jobs showed <1% failure rate over 10 days, reading Parquet from S3 and writing to Iceberg tables via RDS-DMS
- Release combines dynamic attach with Quack and official DuckLake support, completing the local-first lakehouse story but raising open questions about concurrent writer behavior on shared Iceberg tables
The article examines why translating natural language into SQL for live data warehouses remains a tough problem. It traces the rise of ad-hoc queries in 1990s‐era historical data stores and shows how schema complexity and performance needs outpace current text-to-SQL systems.
- Benchmarks like Spider (max 200 tables, single database per task) don't reflect real enterprise warehouses with hundreds of tables and thousands of columns spanning dozens of schemas
- Production metadata is messy: cryptic/inconsistent column naming, typos, abbreviations, and mixed-language query logs undermine model performance
- Vendor-specific SQL extensions (Oracle's LISTAGG, PostgreSQL's JSONB operators) and user-defined types routinely break academic text-to-SQL models
- The authors call for shared benchmarks built from anonymized real industry schemas/workloads and argue LLMs need integration with metadata services and human feedback loops before replacing analysts in critical settings