Click any tag below to further narrow down your results
Links
This article explains state-aware orchestration, a method that enables efficient data pipeline management by tracking the state of tables and their dependencies. It discusses how this approach can reduce unnecessary processing and costs, particularly in complex environments with multiple data sources and schedules.
This article outlines essential lessons for scaling data products, emphasizing the importance of a strong data foundation over complex models. It advocates treating data pipelines like products with clear ownership and standardized processes to enhance reliability and trust in data.