The article examines the performance of several data processing engines—DuckDB, Polars, Daft, and Spark—using a 650 GB dataset in a Delta Lake format on S3. The author expresses frustration with the inefficiencies and costs associated with traditional Spark clusters, labeling this widespread issue as “cluster fatigue.” In contrast, newer tools like DuckDB and Polars promise to handle large datasets effectively without the need for distributed clusters, which can be expensive and complex. The author sets up a test using a single-node EC2 instance with 32GB of RAM and runs each engine to see how they manage the 650 GB dataset. DuckDB completes the task in 16 minutes, showcasing its efficiency and ability to handle deletion vectors. Polars performs slightly better at 12 minutes, though it lacks deletion vector support, which is a significant drawback. Daft, while well-regarded for its performance, struggles with a 50-minute runtime. In comparison, PySpark takes over an hour, highlighting its shortcomings on a single-node setup without fine-tuning. The results illustrate a clear trend: single-node frameworks can match or exceed the performance of traditional distributed systems for specific workloads, potentially allowing users to cut costs. However, the lack of out-of-the-box support for streaming writes in tools like Polars and Daft presents a challenge that needs addressing. The author emphasizes the necessity for these frameworks to evolve and cater to modern data processing needs while alleviating memory pressure.