2 links tagged with all of: data-engineering + optimization
Click any tag below to further narrow down your results
Links
This article explains how to use dbt's defer feature to improve CI/CD pipeline efficiency by only rebuilding modified models instead of the entire project. It covers the setup process, benefits, and potential pitfalls of implementing defer in dbt workflows.
The article discusses a common data engineering exam question focused on optimizing SQL queries with range predicates. It emphasizes adopting a first principles mindset, thinking mathematically about SQL, and using set operations for improved performance. The author provides a step-by-step solution for rewriting a SQL condition to illustrate the benefits of this approach.