My DuckDB + AI Workflow

Open original

The author shares a year of experience using AI for data work, arguing that copying someone else's workflow is pointless — what matters is learning specific techniques. He's settled on DuckDB CLI + Claude as his stack because it reduces hallucinations and keeps agents focused on actual tools instead of generating buggy code.

  • AI workflows are creative, not formulaic — tips and tricks transfer better than full process replication, similar to how watching a music producer's exact steps won't let you recreate their song
  • DuckDB CLI commands paired with AI agents dramatically cut hallucinations because agents understand the tool's actual capabilities rather than inventing Python code that doesn't work
  • Working in a modern terminal (Ghostty) with multiple windows beats IDEs for analytics work — it's faster, gives you exactly the tools you need, and agents are already built to work with CLI tools

The author opens by rejecting the common tech trend of copying someone else's AI workflow, comparing it to music production—you can watch Disclosure explain exactly how they made "Latch," but you won't recreate it by following their steps. The real issue is that AI work is creative and cerebral, not programmatic. What actually matters is learning specific tips and tricks you can adapt to your own process, not replicating someone's entire system. The author uses two different workflows: Cursor Pro for web and software development (building sites and tools like parkbench and fletch), and Claude Pro with Code for analytics work, which replaced their previous Cursor-based approach over the last six months. For analytics specifically, the author found that smaller, cheaper models work fine; MCPs handle SaaS platform interactions; and almost everything can be done in a terminal using command-line text editors like nano. The key insight is that DuckDB's CLI is nearly hallucination-free for AI agents because it forces them to run actual queries instead of generating Python code. Even using an older Haiku model from two years ago, agents understand DuckDB commands well enough to analyze datasets accurately. This matters because most real-world use cases don't involve the enterprise nightmare of scattered PDFs and hundreds of tables—they involve clean gold layer data that AI can actually work with effectively. The author's main advice boils down to staying in the terminal rather than using analytical IDEs. They stopped using any analytical IDE over a year ago, instead running DuckDB queries directly in the system terminal. This shift happened partly through building CLI tools like parkbench and fletch and realizing that terminal-based workflows are where AI agents actually excel. The author suggests upgrading from your default system terminal (mentioning Ghostty as an example) because agents love working with CLI tools, and this approach eliminates the bloat of expensive data platforms while keeping everything fast and local.

Ask about this article