OpenTelemetry is an open-source observability framework designed to provide a standardized way to collect, process, and export telemetry data such as traces, metrics, and logs. It aims to help developers and organizations gain insights into their systems' performance and behavior, facilitating better monitoring and troubleshooting. By integrating with various backend systems, OpenTelemetry enhances observability across diverse environments and applications.
Pinterest's Big Data Platform team has developed Moka, a next-generation data processing platform deployed on AWS Elastic Kubernetes Service (EKS). The article outlines Moka's infrastructure, including its logging and observability strategies, which leverage tools like Fluent Bit for log management and Prometheus for metrics storage and monitoring. Key learnings and future directions for Moka's development are also discussed.