Click any tag below to further narrow down your results
Links
This is a free, searchable directory of 575 open-source projects and companies built on DuckDB, the embedded analytics database. It's rebuilt from the awesome-duckdb GitHub list and updated as of October 2026.
- The map covers 516 open-source projects and 59 companies that publicly say they build on DuckDB, each linked to its source.
- The most-starred projects include PyGWalker (16K stars), Kepler.gl (12.1K), and Lance (7.2K), and the top UIs are Preswald and Online DuckDB Shell.
- Snowflake's own Snowset data shows 96.2% of queries scanned under 10 GB, a size one DuckDB instance can handle, so many workloads could move off a warehouse.
dbt Labs open-sourced dbt Charts, a YAML-based language for building dashboards as code instead of through UI clicks. It's designed to work seamlessly with AI agents and integrates directly with dbt projects in Git.
- Charts declared in YAML can be validated, version-controlled, and changed alongside data models in a single Git branch, catching breaking changes before production.
- AI agents struggle with BI tool UIs but work fluently with code and SQL, so moving charts into YAML gives them a tighter feedback loop with strict validation and visualization warnings.
- The open-source language runs locally and in CI/CD pipelines, while dbtCharts.com adds hosting, access control, and conversational analytics on top without locking users into the platform.
The author shares a year of experience using AI for data work, arguing that copying someone else's workflow is pointless — what matters is learning specific techniques. He's settled on DuckDB CLI + Claude as his stack because it reduces hallucinations and keeps agents focused on actual tools instead of generating buggy code.
- AI workflows are creative, not formulaic — tips and tricks transfer better than full process replication, similar to how watching a music producer's exact steps won't let you recreate their song
- DuckDB CLI commands paired with AI agents dramatically cut hallucinations because agents understand the tool's actual capabilities rather than inventing Python code that doesn't work
- Working in a modern terminal (Ghostty) with multiple windows beats IDEs for analytics work — it's faster, gives you exactly the tools you need, and agents are already built to work with CLI tools
dbt Labs open-sourced a YAML-based charting language that lets AI agents generate dashboards as code instead of UI-bound reports, solving the friction between messy code generation and restrictive BI tool interfaces. The accompanying dbtCharts.com platform adds hosting, access control, and conversational analytics on top of the open standard.
- Current AI-generated reports create sprawling file structures across multiple languages, making audits slow and token-expensive; dbt Charts consolidates everything into a single readable YAML file that agents can modify efficiently
- Charts defined in code live in Git alongside dbt models, so schema changes and chart updates ship together in one CI run and fail before reaching production
- The language includes 1,100+ config options across 16 chart types with cascading styles and inheritance, designed to look intentionally crafted rather than dashboard-grid generic
xlDuckDb is an Excel add-in that lets you write DuckDB SQL queries directly in spreadsheet cells and get results back as normal Excel data. You can query Excel ranges, JSON files, CSV files, Parquet files, and even remote data on HTTPS or AWS S3.
- The add-in works with Excel ranges, named ranges, and tables using simple syntax like =DuckDbQuery(A27,,A1:E25), plus you can query multiple ranges in a single query by passing them as separate arguments.
- DuckDB handles messy data formats automatically—it auto-detects CSV delimiters and data types, extracts nested JSON with JSONPath syntax, and efficiently queries large Parquet files that don't fit in memory.
- You can join data across multiple file types in one SQL query (e.g., combining JSON and CSV data with a CTE and INNER JOIN), making it possible to correlate disparate datasets without leaving Excel.
AWS acquired DuckLabs—the team behind the fast-growing DuckDB analytics database—to control the roadmap of a key data infrastructure technology and build an S3-powered alternative to Databricks and Snowflake. The deal doesn't include the open-source DuckDB foundation or MotherDuck, just the core engineering team and their vision.
- DuckDB, DuckLake, and Quack form a free, composable data stack that runs on AWS's S3 storage; AWS profits from increased compute and storage usage even when customers skip the vendor tax of proprietary platforms.
- AI agents need lightweight, embeddable databases for managing state and telemetry at scale, and DuckDB's architecture—small, fast-starting, process-embedded—makes it ideal for this emerging workload.
- AWS likely paid hundreds of millions or close to $1 billion, similar to Databricks's $1B+ acquisition of Tabular; the value lies in Hannes Mühleisen and Mark Raasveldt's vision and ability to shape DuckDB's evolution toward becoming a general-purpose distributed database.
AWS bought DuckLabs (the team behind DuckDB) to control the roadmap of a fast-growing analytics database that's becoming foundational to modern data infrastructure. The deal is really about acquiring talented engineers and influence over DuckDB's evolution toward becoming a distributed, server-based system that competes with Databricks and Snowflake.
- DuckDB + DuckLake + Quack form an open, S3-powered data stack that lets AWS compete with Databricks and Snowflake while making money on cloud primitives (compute, storage, networking) rather than vendor markup.
- DuckDB is well-suited for AI agents because it's lightweight, embeddable, starts instantly, and handles analytics on telemetry streams—exactly what agents need for self-optimization and parallel task exploration.
- AWS likely paid hundreds of millions to over $1 billion (comparable to Databricks' $1B+ Tabular deal) because even modest improvements to AWS's competitive position in data infrastructure justify massive payouts.
AnalystGym offers a two-minute daily drill focused on analytical judgment rather than technical skills—the part of data work that AI can't handle. You rate your confidence, get feedback on your reasoning, and build a streak by showing up consistently.
- AI handles execution well (queries, transformations, calculations), but you still own the judgment calls: interpreting sudden metric changes, handling stakeholder requests, and defining metrics that hold up under pressure
- Four rotating drill formats (Lesson, Rapid Fire, Currency, The Rule) train different skills: careful reading, pattern recognition, time budgeting, and precise rule-writing
- Getting answers wrong is part of the learning—the value is in the reps and calibrating your confidence, not in being right every time
Apache Ossie is an Apache Incubator project that defines a vendor-neutral YAML spec for semantic data models. It lets teams declare metrics, dimensions and joins once and share them across BI, analytics and AI tools. This prevents metric drift, cuts integration debt and creates a single source of truth.
- Apache Ossie standardizes metric/dimension definitions in YAML so BI, analytics, and AI tools all reference one shared source of truth instead of redefining metrics per dashboard.
- Over 50 organizations, including Snowflake, Databricks, Oracle, Salesforce, dbt Labs, Qlik, and ThoughtSpot, have joined the effort.
- The spec was renamed from Open Semantic Interchange to Apache Ossie in July 2026, shortly after launching a Financial Services Semantic Working Group in June 2026.
- It embeds AI context instructions so LLMs can ground answers in official business logic, aiming to cut reconciliation costs and eliminate conflicting dashboards.
Databricks introduced Lakehouse//RT—powered by the new Reyden engine—to deliver millisecond analytics directly on Delta Lake, removing the need for separate serving databases. The summit also revealed LTAP, which merges OLTP and OLAP on a single data copy, and Lakebase, a PostgreSQL-compatible engine, signaling a push to unify transactional, analytical, and real-time workloads.
- Lakehouse//RT (via the Reyden engine) claims 10–100ms query latency and ~12,000 QPS directly on Delta Lake, aiming to eliminate separate serving databases like ClickHouse or Pinot.
- LTAP merges OLTP and OLAP on a single data copy, and Lakebase adds a Postgres-compatible engine, pushing toward unifying transactional and analytical workloads.
- Most of the new AI-agent tools (Genie Ontology, ZeroOps, Omnigent, Unity AI Gateway) are dismissed as undifferentiated experiments, with Databricks' real edge seen in core infrastructure, not agent frameworks.
- The bigger bet is architectural: killing data copies, sync pipelines, and duplicated security rules rather than just raw speed gains.
AgentsView is a self-hosted tool that indexes your AI coding agent sessions into a local SQLite or PostgreSQL store and serves a web UI on localhost. It tracks token usage and compute costs across multiple agents, offers CLI commands for usage reports, and supports Docker, desktop apps, and background server modes. It also provides full-text search, analytics dashboards, and per-session or daily cost breakdowns.
- AgentsView is a self-hosted, local-first tool that indexes AI coding agent sessions (Claude, Codex, Forge, OpenCode, and 20+ others) into SQLite/DuckDB/PostgreSQL with no cloud dependency.
- The CLI's `agentsview usage` command is positioned as a replacement for ccusage, giving sub-second token/cost breakdowns by model, date, or agent.
- It supports flexible deployment (install script, Homebrew, Docker, background daemon) with production-ready configs like read-only mounts, PostgreSQL backing, and optional public exposure via `--public-url`/`--require-auth`.
- Beyond raw usage stats, `agentsview stats` adds higher-level analytics like session archetypes (quick/deep/marathon) and optional Git-linked outcomes (commits, PRs).
Andy Warfield recounts how genomic researchers struggled with moving large datasets between local filesystems and S3, spurring the creation of S3 Files to present S3 buckets as POSIX-compatible file stores. The post then covers how Amazon added S3 Tables for managed Iceberg tables and built-in vector indexing to cut storage friction and speed up analytics and search.
- S3 Files was inspired by genomics researchers at UBC who wasted hours copying sequencing data between NFS and S3 because their GATK4/Spark pipeline still required POSIX filesystem semantics despite massive serverless parallelism.
- Agent-driven development is lowering the barrier for domain experts to build apps, making it more critical to decouple persistent data from any single application or workflow.
- S3 handles over 25 million Parquet requests daily, often via Hive tables, but open formats like Iceberg still force customers to manage object-level APIs, compaction jobs, and metadata services themselves.
- S3 Files aims to provide a filesystem-style interface on top of S3's scale and durability, eliminating manual copy scripts and unifying table-format intelligence under one system.
mviz is a Claude skill that simplifies the creation of static reports for ad hoc data analysis by converting compact JSON specifications into professional HTML visualizations. It emphasizes a fast, AI-driven workflow that allows users to iterate quickly, generate reports, and utilize a variety of chart types without extensive coding. The tool works seamlessly with data from various sources, including local files and cloud databases.
- Mviz is a Claude skill that turns compact JSON specs into polished HTML reports/charts, prioritizing speed over BI-style reusability.
- Workflow: build context via natural-language queries (Claude writes the SQL), form a narrative, generate an initial chart, then iterate with follow-up commands.
- Supports many chart types (bar, line, heatmaps, funnels, tables, KPIs) laid out in a customizable 16-column grid.
- Can pull data directly from local files or cloud databases, producing clean, responsive output suitable for print or digital sharing.
Excluding your internal IP address from Google Analytics 4 is essential for maintaining accurate website data analytics. The article outlines the steps to filter out your IP address, ensuring that your data is not skewed by internal traffic, particularly important as Universal Analytics transitions to GA4.
- GA4 doesn't collect data retroactively, so a misconfigured IP filter can mean permanently lost data—test it before relying on it.
- Setting up the exclusion involves defining internal traffic and creating a filter in GA4 settings to keep internal/vendor visits from skewing reports.
- IP addresses can change with location, so filters need periodic updates to stay accurate.
- Universal Analytics was decommissioned July 1, 2023, pushing marketers to migrate to GA4 to keep tracking traffic effectively.
Universal Analytics offers an IP-address masking feature that truncates user IP addresses to enhance privacy and comply with data protection regulations. The process occurs in memory before any data is stored, ensuring full IP addresses are never recorded. Website owners can implement this feature using the anonymize_ip configuration in the gtag.js library.
- IP masking zeroes out the last octet of IPv4 addresses and the last 80 bits of IPv6 addresses almost immediately after Google Analytics receives the data.
- Truncation happens in memory before any storage occurs, so the full IP address is never written to disk.
- Enabling the feature only requires setting the anonymize_ip parameter (aip=1) via gtag.js.
- Even with masking, enough geographic information survives in the truncated address to still support location-based reporting.
Livedocs is a collaborative platform that merges the functionality of notebooks with app-building simplicity, ideal for various data tasks such as exploration, analysis, and visualization. It supports powerful AI tools, enabling users to perform advanced analytics, create interactive dashboards, and share insights effortlessly.
- Livedocs combines notebook-style flexibility with app-building simplicity for data exploration, analysis, and visualization
- Built-in AI tools enable advanced analytics and let users build interactive dashboards without heavy manual coding
- The platform emphasizes collaboration, making it easy to share insights and outputs with others
The text appears to be corrupted and unreadable, making it impossible to extract coherent content or information about the topic. As a result, no summary can be provided due to the lack of accessible details.
- The original article content was corrupted/unreadable, so this is based solely on the provided summary/metadata
- The book, by Yusuf Aytas, is 336 pages and targets data analysts, engineers, scientists, and students building Big Data platforms
- It covers stream processing, analytics, data science, discovery, and security, with step-by-step guidance from basic scripting to distributed systems