Apache DataFusion 50.0.0 has been released, featuring significant performance enhancements, including improved dynamic filter pushdown and nested loop join optimizations. The update introduces new capabilities such as support for the QUALIFY SQL clause and extended functionality for window functions, alongside community growth and contributions.
Apache Spark 4.0.0 is the first release in the 4.x series, showcasing significant community collaboration with over 5100 resolved tickets. Major enhancements include a new lightweight Python client, expanded features in Spark SQL and PySpark, and improved structured streaming capabilities, alongside numerous other updates for better performance and usability.