AWS has introduced the MCP Server for Apache Spark History Server, enabling AI-driven debugging and optimization of Spark applications by allowing engineers to interactively query performance data using natural language. This open-source tool simplifies the traditionally complex process of performance troubleshooting, reducing the reliance on deep technical expertise and manual workflows. The MCP Server integrates seamlessly with existing Spark infrastructures, enhancing observability and operational efficiency.

Amazon SageMaker's lakehouse architecture now automates the optimization of Apache Iceberg tables on Amazon S3, simplifying maintenance through catalog-level configuration. This enhancement allows data lake administrators to enable automated table optimizations, such as compaction and orphan file deletion, across all Iceberg tables with a single setting, improving performance and cost efficiency.