The author shares a year of experience using AI for data work, arguing that copying someone else's workflow is pointless — what matters is learning specific techniques. He's settled on DuckDB CLI + Claude as his stack because it reduces hallucinations and keeps agents focused on actual tools instead of generating buggy code.

• AI workflows are creative, not formulaic — tips and tricks transfer better than full process replication, similar to how watching a music producer's exact steps won't let you recreate their song
• DuckDB CLI commands paired with AI agents dramatically cut hallucinations because agents understand the tool's actual capabilities rather than inventing Python code that doesn't work
• Working in a modern terminal (Ghostty) with multiple windows beats IDEs for analytics work — it's faster, gives you exactly the tools you need, and agents are already built to work with CLI tools

DuckDB released a plugin that lets Claude Code query data files directly using SQL instead of writing Python scripts, making it faster and more accurate. The plugin handles everything from local CSVs to remote cloud storage and spatial data, with Claude automatically picking the right tool based on what you ask.

• Claude can now run SQL queries against any file format (CSV, Parquet, JSON, Excel, etc.) on disk or in cloud storage, getting exact answers instead of guessing column names
• The plugin includes specialized skills for S3/GCS exploration, spatial queries (distances, nearest neighbors), searching DuckDB docs, and recalling decisions from past sessions
• When a query fails, Claude reads the error message and retries with corrected SQL automatically, creating a two-layer conversation (natural language with you, SQL with DuckDB)

xlDuckDb is an Excel add-in that lets you write DuckDB SQL queries directly in spreadsheet cells and get results back as normal Excel data. You can query Excel ranges, JSON files, CSV files, Parquet files, and even remote data on HTTPS or AWS S3.

• The add-in works with Excel ranges, named ranges, and tables using simple syntax like =DuckDbQuery(A27,,A1:E25), plus you can query multiple ranges in a single query by passing them as separate arguments.
• DuckDB handles messy data formats automatically—it auto-detects CSV delimiters and data types, extracts nested JSON with JSONPath syntax, and efficiently queries large Parquet files that don't fit in memory.
• You can join data across multiple file types in one SQL query (e.g., combining JSON and CSV data with a CTE and INNER JOIN), making it possible to correlate disparate datasets without leaving Excel.

AnalystGym offers a two-minute daily drill focused on analytical judgment rather than technical skills—the part of data work that AI can't handle. You rate your confidence, get feedback on your reasoning, and build a streak by showing up consistently.

• AI handles execution well (queries, transformations, calculations), but you still own the judgment calls: interpreting sudden metric changes, handling stakeholder requests, and defining metrics that hold up under pressure
• Four rotating drill formats (Lesson, Rapid Fire, Currency, The Rule) train different skills: careful reading, pattern recognition, time budgeting, and precise rule-writing
• Getting answers wrong is part of the learning—the value is in the reps and calibrating your confidence, not in being right every time

OpenAI’s data productivity team built Kepler, an AI agent that understands their data platform context and writes iterative SQL to answer complex data questions. It handles table discovery, query refinement, follow-up prompts and even visualizations, cutting analysis time and reducing errors.

• OpenAI built Kepler, an internal AI agent that writes iterative SQL to answer data questions across 70,000 datasets and 600 petabytes daily, deployed via Slack, IDEs, or web UI
• It doesn't just query once—it discovers tables, refines CTEs, verifies numbers against dashboards/Notion, and proposes hypotheses (e.g., duplicate logging) for anomalies like a WAU spike
• Memory for self-learning plus rigorous evals let it improve over time without regressing
• Rolled out to 80% of the company, turning multi-engineer, multi-hour investigations into minutes

Hex built a suite of analytical evals to test data-analysis models and found Claude Fable 5 outperforms its Opus 4.x predecessors by 10–15%, nailing both semantically modeled and raw-data tasks with fewer mistakes. They’ve also designed a tougher “Frontier” benchmark for long-horizon, open-ended scenarios, where Fable 5’s careful assumptions and cross-checks boost its pass rate to around 58%.

• Claude Fable 5 beats Opus 4.7 by 10-15 points on Hex's core benchmarks, scoring 93%+ on Analytical Hard/Semantically Modeled tests and 65% on Semantically Unmodeled tasks, versus prior Opus versions' single-digit gains
• Fable's advantage comes from following a "golden workflow" (starting in the semantic layer, cross-checking raw SQL) and transparently stating assumptions, which lets it catch errors like a cents-for-dollars mistake that Opus misses
• On Hex's new "Frontier" benchmark for long-horizon, open-ended tasks, Fable at Max Effort hits 58% pass rate, notably outperforming other setups

The article discusses the shortcomings of achieving high accuracy in Text-to-SQL systems, emphasizing that 90% accuracy is insufficient for enterprise applications. It highlights the need for rigorous evaluation frameworks, like Spider 2.0, to ensure reliability and trust in AI-driven analytics.

• 90% accuracy still means 1 in 10 SQL queries is wrong, which is enough to destroy user trust and stall enterprise adoption.
• Standard benchmarks like Spider 1.0 don't reflect enterprise reality; Spider 2.0 tests against schemas with 3,000+ columns to simulate real-world messiness.
• Simple accuracy scores are misleading — metrics like Execution Accuracy (EX) and Soft-F1 are needed to actually capture whether generated SQL is reliable.

The author shares their shift from using Excel and Google Sheets to DuckDB and SQL for handling CSV files, highlighting the efficiency of querying data directly. They discuss the benefits of using SQL for data manipulation and invite readers to share their own CSV handling tips.

• DuckDB lets you run SQL directly on CSV files (joins, window functions, dedup queries) without importing them into Excel/Sheets first
• You can query CSVs from URLs or cloud storage like S3 directly, and export results straight back to CSV
• Several users find SQL queries more intuitive than replicating the same logic with spreadsheet formulas or pivot tables
• Some commenters mix in Parquet files and other extensions alongside DuckDB to further speed up ad-hoc analysis