Pinterest is transitioning from its aging Hadoop-based platform to a Kubernetes-based data processing solution named Moka, designed to address scalability and performance needs. The first part of this series discusses the rationale behind this shift, the architecture of the new platform, and initial design considerations, while outlining the benefits of using Kubernetes for data processing at massive scale.

Pinterest's Big Data Platform team has developed Moka, a next-generation data processing platform deployed on AWS Elastic Kubernetes Service (EKS). The article outlines Moka's infrastructure, including its logging and observability strategies, which leverage tools like Fluent Bit for log management and Prometheus for metrics storage and monitoring. Key learnings and future directions for Moka's development are also discussed.