Click any tag below to further narrow down your results
Links
DuckDB released a plugin that lets Claude Code query data files directly using SQL instead of writing Python scripts, making it faster and more accurate. The plugin handles everything from local CSVs to remote cloud storage and spatial data, with Claude automatically picking the right tool based on what you ask.
- Claude can now run SQL queries against any file format (CSV, Parquet, JSON, Excel, etc.) on disk or in cloud storage, getting exact answers instead of guessing column names
- The plugin includes specialized skills for S3/GCS exploration, spatial queries (distances, nearest neighbors), searching DuckDB docs, and recalling decisions from past sessions
- When a query fails, Claude reads the error message and retries with corrected SQL automatically, creating a two-layer conversation (natural language with you, SQL with DuckDB)
Apache Spark 4.1.0 adds declarative pipelines, real-time streaming mode with sub-second latency, improved PySpark performance, and SQL scripting as a stable feature. The release resolved over 1,800 issues with contributions from more than 230 developers.
- Spark Declarative Pipelines (SDP) lets you define datasets and queries while Spark handles execution graphs, parallelism, checkpoints, and retries automatically.
- Structured Streaming Real-Time Mode enables continuous processing with sub-second latency for stateless tasks, dropping to single-digit milliseconds in some cases.
- Arrow-native PySpark UDFs and UDTFs eliminate Pandas conversion overhead, and Python Data Sources now support filter pushdown to reduce data movement.
- SQL Scripting is now GA and enabled by default, while the VARIANT type (for semi-structured data) is GA with shredding support for faster reads.
OpenAI’s data productivity team built Kepler, an AI agent that understands their data platform context and writes iterative SQL to answer complex data questions. It handles table discovery, query refinement, follow-up prompts and even visualizations, cutting analysis time and reducing errors.
- OpenAI built Kepler, an internal AI agent that writes iterative SQL to answer data questions across 70,000 datasets and 600 petabytes daily, deployed via Slack, IDEs, or web UI
- It doesn't just query once—it discovers tables, refines CTEs, verifies numbers against dashboards/Notion, and proposes hypotheses (e.g., duplicate logging) for anomalies like a WAU spike
- Memory for self-learning plus rigorous evals let it improve over time without regressing
- Rolled out to 80% of the company, turning multi-engineer, multi-hour investigations into minutes
This issue covers a new AI-built SQL parser for PostHog that’s 70× faster, Linear’s move from styled-components to StyleX for better React performance, and a deep dive into misleading database benchmarks. It also highlights tools like Gemini 3.5 Flash’s computer use feature, the peerd automation extension, a breakdown of prompt injection vulnerabilities, and other AI and dev updates.
- PostHog built an AI-generated SQL parser that's 70x faster than their old C++ ANTLR version, using property-based tests to guide the model, without changing the query language for users
- Linear is replacing styled-components with StyleX to reduce runtime style recalculations and cut style-sheet overhead, since styled-components hasn't kept up with React
- LLMs can't reliably distinguish system/user role tags because they follow tone rather than tag hierarchy, letting attackers embed commands like "ignore previous" inside user-level text to bypass safeguards
This tool parses your SQL CREATE TABLE and ALTER TABLE statements and instantly renders an interactive ER diagram in your browser. It works entirely locally—no uploads or accounts—supports PostgreSQL, MySQL, SQLite, and SQL Server, and lets you drag tables, rename elements, and export to PNG or SVG.
- Paste SQL DDL (CREATE/ALTER, across Postgres/MySQL/SQLite/SQL Server) and get an instant interactive ER diagram, fully client-side with no uploads or accounts.
- Diagrams are editable—drag tables, rename fields, auto-layout—and correctly detect not-null, unique, and key/foreign-key constraints.
- Export options include high-res PNG, vector SVG, saved project files, or a shareable URL that embeds the diagram itself.
- Free, open source, installation-free, and works across desktop and mobile.
ggsql extends SQL so you can write data queries and visualization specs in one composable syntax based on the Grammar of Graphics. It compiles to WebAssembly, works with DuckDB/SQLite and Vega-Lite, and offers an in-browser playground for immediate testing.
- ggsql lets you write a SQL query and a Grammar-of-Graphics chart spec (VISUALISE/DRAW/SCALE/LABEL) in one statement, skipping export-then-plot workflows in Python/R
- It compiles to WebAssembly, runs in-browser via a playground, and currently works with DuckDB/SQLite for data and Vega-Lite for rendering (alpha stage, more connectors/exports planned)
- Its simple, declarative key-value syntax is designed to be easy for both humans and AI tools to generate and inspect
PgQue is a pure SQL/PLpgSQL event queue for Postgres 14+ that delivers durable, Kafka-style streams without dead tuples or external daemons. It uses snapshot-based batching and TRUNCATE-driven table rotation for stable performance under sustained load. A pg_cron (or any scheduler) ticker drives 1–2 second end-to-end delivery.
- Uses snapshot-based batching with TRUNCATE-driven table rotation instead of per-row UPDATE/DELETE with SKIP LOCKED, avoiding dead tuples, VACUUM pressure, and bloat under sustained load.
- Pure SQL/PLpgSQL install (single file) with pg_cron ticking every second, so it runs on managed Postgres without C extensions or custom builds—unlike PgQ, PGMQ, River, Que, and pg-boss.
- Offers Kafka-style shared event logs with independent per-consumer cursors, rather than one-job-one-worker task queue semantics.
- Delivery latency is 1–2 seconds (tick interval plus polling), with microsecond function execution—fine for durability-focused use but not for sub-millisecond dispatch needs.
Rill’s Metrics SQL lets you define business metrics once and query them using plain SQL across dashboards, notebooks, and AI agents. It compiles metric views into optimized OLAP queries, handling grouping, filters, time functions, and security automatically.
- Define a metric once in a YAML-backed metrics view and query it as plain SQL from dashboards, notebooks, and AI agents alike, instead of redefining it in dbt, Looker, Python, etc.
- A parser/compiler/executor pipeline rewrites simple queries like "SELECT country, revenue FROM revenue_metrics" into full OLAP SQL with correct GROUP BY, joins, and filters automatically.
- Security (row-level and column permissions) and SQL-injection-safe parameter binding are enforced consistently at the executor stage for every consumer.
- Works across multiple backends (ClickHouse, Snowflake, DuckDB, Druid) and can leverage existing database optimizations like materialized views and indexes for performance.
The author shares their shift from using Excel and Google Sheets to DuckDB and SQL for handling CSV files, highlighting the efficiency of querying data directly. They discuss the benefits of using SQL for data manipulation and invite readers to share their own CSV handling tips.
- DuckDB lets you run SQL directly on CSV files (joins, window functions, dedup queries) without importing them into Excel/Sheets first
- You can query CSVs from URLs or cloud storage like S3 directly, and export results straight back to CSV
- Several users find SQL queries more intuitive than replicating the same logic with spreadsheet formulas or pivot tables
- Some commenters mix in Parquet files and other extensions alongside DuckDB to further speed up ad-hoc analysis
Since the inception of SQL in 1974, there has been a recurring dream to replace data analytics developers with tools that simplify the querying process. Each decade has seen innovations that aim to democratize data access, yet the complex intellectual work of understanding business needs and making informed decisions remains essential. Advances like AI can enhance efficiency but do not eliminate the crucial human expertise required in data analytics.
- SQL was designed in 1974 by IBM's Chamberlin and Boyce specifically to let non-programmers query data in plain English, yet this self-service dream has resurfaced every decade (OLAP in the 80s, semantic layers now) without ever eliminating the need for developers.
- AI can now generate SQL and build analytical models faster than ever, but it automates only the mechanical coding, not the judgment calls about which metrics or definitions actually matter.
- The real bottleneck was never syntax or tooling but the intellectual work of understanding business context — a gap AI accelerates around but doesn't close.
chDB transforms ClickHouse into a user-friendly Python library for seamless DataFrame operations, eliminating serialization overhead and enabling fast SQL queries directly on Pandas DataFrames. The latest version achieves significant performance improvements, making it 87 times faster than its predecessor by implementing zero-copy data handling and optimized processing.
- chDB now queries Pandas DataFrames directly via zero-copy access, eliminating serialization overhead entirely.
- The latest version is 87x faster than its predecessor.
- Optimized processing combined with zero-copy handling makes chDB the fastest SQL engine benchmarked on DataFrames.