Click any tag below to further narrow down your results
+ observability
(2)
+ machine-learning
(2)
+ data-architecture
(2)
+ responsible-ai
(1)
+ sql-automation
(1)
+ llms
(1)
+ automation
(1)
+ incident-management
(1)
+ data-mesh
(1)
+ risk-management
(1)
+ metrics
(1)
+ semantic-layer
(1)
+ data-governance
(1)
+ personalization
(1)
+ load-balancing
(1)
Links
A 378-page book teaching practical approaches to building ML models by prioritizing data quality over algorithm complexity, covering data collection, cleaning, labeling, and synthetic data generation with Python. Published February 2024, it emphasizes responsible AI and the role of subject-matter experts in model development.
- Shifts focus from algorithm optimization to data quality as the foundation for robust, fair, and interpretable ML models
- Covers concrete techniques: data imputation, cleaning, labeling, augmentation, and synthetic data generation with scikit-learn code examples
- Introduces "small data" concept and strategies for handling missing data, addressing bias, and building ethical AI systems
A review of "Data-Centric Machine Learning with Python" arguing that ML success depends more on data quality than model sophistication, and explaining why this approach has become standard practice. The reviewer extends this to show how LLMs fit into data-centric pipelines as both label generators and sources of weak supervision.
- The shift to data-centric ML happened years ago through landmark projects like Netflix Prize, ImageNet, and Word2Vec—not a recent invention—driven by the internet's planet-scale data generation.
- Data preparation is the real bottleneck in modern ML: cleaning, imputing missing values, extracting features, removing bias, and iteratively refining labels through human-in-the-loop active learning takes far more effort than training.
- LLMs can be treated as replacements for human subject matter experts in labeling and weak supervision, which means the techniques for managing SME bias and inconsistency now apply directly to LLM outputs.
Grab built an automated system called Data Production Issues (DPIs) that detects when certified data products fail their contracts, diagnoses root causes without manual intervention, and fixes common failures automatically. The system keeps data quality issues actionable by filtering noise, assigning problems to the right owner, and closing issues only when the underlying data actually recovers.
- DPIs use a standardized Data Health API with four error categories (upstream, platform, job, data) to route failures to the correct team without requiring platform-specific knowledge, enabling consistent incident management across Grab's heterogeneous data systems.
- The triage phase deduplicates alerts and groups related failures by root cause, preventing responders from drowning in repetitive notifications while ensuring only real contract breaches become operational issues.
- Hugo, Grab's ingestion platform, consolidates signals from multiple sources into a three-layer diagnosis workflow that automatically identifies root causes and recommends fixes, eliminating manual log inspection and shortening remediation time.
Organizations face three recurring data problems—inconsistent metric definitions, fragmented access controls, and metric changes that don't propagate everywhere. A semantic layer solves this by centralizing metric definitions and governance in one place, so all tools pull the same numbers and changes cascade automatically. It won't fix bad data at the source, but it shrinks the surface area you need to manage and makes self-service analytics actually work.
- The same metric getting defined differently across Tableau, Power BI, and Python isn't a minor annoyance—it's a root cause of bad executive decisions.
- Hiring more BI analysts as gatekeepers just creates ticket queues and bottlenecks; it doesn't fix fragmented governance across tools.
- A semantic layer centralizes metric definitions and access controls so a single change (e.g., redefining ARR) propagates automatically everywhere instead of requiring manual updates across systems.
- Centralizing definitions and logic in one place also makes the data self-documenting, enabling real self-service instead of ticket-based requests.
This roundup covers Netflix’s switch to Kueue for Kubernetes-native batch compute, an engineer’s workflow for long-running coding agents, and Zalando’s in-process client load balancer handling over a million requests per second. It also explains Zepto’s dual-sequence re-ranker for real-time personalization, strategies for catching data issues early, why technically strong teams still miss business impact, and a new storage/workload architecture taxonomy—plus a Databricks metrics webinar and SQL tools.
- Netflix moved its batch compute platform to Kueue, mapping its tenant hierarchy to Cohorts/ClusterQueues/LocalQueues and adding preemption-based fair sharing so critical jobs run without manual intervention.
- Zalando built an in-process client-side load balancer for its Product Read API that handles over a million requests per second, matching Skipper's consistent hashing via Kubernetes watch-based discovery and AZ-aware N-ring fade-in.
- Zepto's Dual Sequence ReRanker uses separate transformer encoders for long-term history and in-session behavior, rebuilding per-candidate user profiles with target-aware pooling and a learned fusion gate incorporating real-time signals like trending counters.
- Technically strong data teams still fail to drive business impact if they only deliver data without pushing decisions, regardless of skill level.
This article walks through setting up a future-proof data stack by outsourcing ingestion to turnkey tools, sticking with SQL-based transforms, and carefully integrating AI without ignoring fundamentals. It also covers key areas like data quality, storage and compute choices, and when to move from replicas to a full warehouse or lakehouse.
- Ingestion is a solved problem now—use managed connectors (Estuary, Fivetran, Glue, DLT) instead of building custom pipelines unless there's truly no supported option.
- SQL remains the backbone of transformation work regardless of tool (dbt, stored procedures, Airflow), with AI only starting to help via auto-generated schemas and template joins, not replacing hands-on cleanup.
- Flashy AI chatbots/analytics tools often demand as much setup as traditional dashboards, so lightweight CLI or skills-file integrations with tools like Claude or Codex are a more practical automation win.
- Don't over-architect storage/compute—a read replica and plain SQL can beat a lakehouse or warehouse if your team size and actual workload don't justify the complexity.
Writing SQL queries is straightforward, but creating a reliable system for running them efficiently is complex and often results in poor data quality and operational inefficiencies. Transitioning from ad-hoc scripts to a structured, spec-driven architecture enhances reproducibility, validation, and observability of SQL jobs, ultimately leading to better management of data and costs.
- Ad-hoc SQL scripts scale poorly because they lack reproducibility, validation, and observability, leading to silent data quality issues and wasted compute costs.
- Moving to a spec-driven architecture (defining queries as versioned, declarative configs rather than one-off scripts) makes jobs testable, auditable, and easier to rerun consistently.
- Centralizing query execution through a structured system enables built-in validation and monitoring, catching errors before they propagate downstream.