Patreon faced challenges in scaling its infrastructure for live events, necessitating cross-team collaboration to quantify capacity and optimize performance. Through careful analysis and prioritization of app requests, they focused on reducing load and enhancing user experience while maintaining system reliability. Key learnings emphasized the importance of optimizing both client and server aspects to achieve scalability.

Charlotte Qi discusses the challenges of serving large language models (LLMs) at Meta, focusing on the complexities of LLM inference and the need for efficient hardware and software solutions. She outlines the critical steps to optimize LLM serving, including fitting models to hardware, managing latency, and leveraging techniques like continuous batching and disaggregation to enhance performance.

Pinterest has enhanced its machine learning (ML) infrastructure by extending the capabilities of Ray beyond just training and inference. By addressing challenges such as slow data pipelines and inefficient compute usage, Pinterest implemented a Ray-native ML infrastructure that improves feature development, sampling, and labeling, leading to faster, more scalable ML iteration.