BigQuery has introduced significant enhancements for generative AI inference, improving scalability, reliability, and usability. New functions like ML.GENERATE_TEXT and ML.GENERATE_EMBEDDING offer increased throughput, with over 100x gains for LLM models, while reliability boasts over 99.99% success rates. Usability improvements streamline connection setups and automatic quota management, making it easier for users to leverage AI capabilities directly in BigQuery.

Klaviyo successfully migrated its event processing pipeline from RabbitMQ to a Kafka-based architecture, handling up to 170,000 events per second while ensuring zero data loss and minimal impact on ongoing operations. The new system enhances performance, scales for future growth, and improves operational efficiency, positioning Klaviyo to meet the demands of over 176,000 businesses worldwide. Key design principles focused on decoupling ingestion from processing, eliminating blocking issues, and ensuring reliability in the face of transient failures.