Click any tag below to further narrow down your results
Links
AI is creating more work for data teams without corresponding budget increases, forcing engineers to maintain infrastructure instead of building. The article walks through architecture decisions and tool choices that let small teams scale operations without collapsing under demand.
- Only 30% of companies increased data budgets in 2025, yet expectations for governance, observability, and self-service features keep rising—most teams must deliver more with the same headcount
- Data volumes and citizen automations (via tools like n8n) multiply dependencies faster than teams can manage them; without proper orchestration and visibility, this creates invisible work that scales directly with data growth
- Pick infrastructure built for scale (Snowflake, Databricks) paired with a single control plane (like Orchestra) rather than a modular stack that requires constant manual intervention and cluster management
This roundup covers Netflix’s switch to Kueue for Kubernetes-native batch compute, an engineer’s workflow for long-running coding agents, and Zalando’s in-process client load balancer handling over a million requests per second. It also explains Zepto’s dual-sequence re-ranker for real-time personalization, strategies for catching data issues early, why technically strong teams still miss business impact, and a new storage/workload architecture taxonomy—plus a Databricks metrics webinar and SQL tools.
- Netflix moved its batch compute platform to Kueue, mapping its tenant hierarchy to Cohorts/ClusterQueues/LocalQueues and adding preemption-based fair sharing so critical jobs run without manual intervention.
- Zalando built an in-process client-side load balancer for its Product Read API that handles over a million requests per second, matching Skipper's consistent hashing via Kubernetes watch-based discovery and AZ-aware N-ring fade-in.
- Zepto's Dual Sequence ReRanker uses separate transformer encoders for long-term history and in-session behavior, rebuilding per-candidate user profiles with target-aware pooling and a learned fusion gate incorporating real-time signals like trending counters.
- Technically strong data teams still fail to drive business impact if they only deliver data without pushing decisions, regardless of skill level.
This newsletter rounds up deep dives on modern data tooling—from Fable 5’s performance on complex analytic tasks to Airbnb’s flexible multi-product data architecture and PostgreSQL Anonymizer’s new local differential privacy features. It also links to articles on lakehouse governance with Gravitino, vector storage advances, streaming runtimes, and orchestration reactions to Dagster’s price hike.
- Claude Fable 5 beats other frontier models by 10–15% on Hex's messy, long-horizon eval sets designed to test judgment and assumption clarity.
- Airbnb's offline data pipeline enforces plain-vanilla modeling, uniform IDs, and strict namespaces to let teams build domain-specific features without breaking cross-product consistency.
- PostgreSQL Anonymizer 3.1 adds true Local Differential Privacy via GRRM, letting users tune epsilon for surveys and categorical data alongside six PII-masking methods.
- Capital One cut cloud-dormancy analysis from nine months to ten days using AI-generated Spark SQL with human validation.
This article walks through setting up a future-proof data stack by outsourcing ingestion to turnkey tools, sticking with SQL-based transforms, and carefully integrating AI without ignoring fundamentals. It also covers key areas like data quality, storage and compute choices, and when to move from replicas to a full warehouse or lakehouse.
- Ingestion is a solved problem now—use managed connectors (Estuary, Fivetran, Glue, DLT) instead of building custom pipelines unless there's truly no supported option.
- SQL remains the backbone of transformation work regardless of tool (dbt, stored procedures, Airflow), with AI only starting to help via auto-generated schemas and template joins, not replacing hands-on cleanup.
- Flashy AI chatbots/analytics tools often demand as much setup as traditional dashboards, so lightweight CLI or skills-file integrations with tools like Claude or Codex are a more practical automation win.
- Don't over-architect storage/compute—a read replica and plain SQL can beat a lakehouse or warehouse if your team size and actual workload don't justify the complexity.