BigQuery has introduced significant enhancements for generative AI inference, improving scalability, reliability, and usability. New functions like ML.GENERATE_TEXT and ML.GENERATE_EMBEDDING offer increased throughput, with over 100x gains for LLM models, while reliability boasts over 99.99% success rates. Usability improvements streamline connection setups and automatic quota management, making it easier for users to leverage AI capabilities directly in BigQuery.

The article explores the challenges of measuring the usage and impact of generative AI, highlighting the confusion around metrics such as active users and token generation. It draws parallels to early internet metrics while emphasizing the need for clarity in definitions and understanding the context of AI adoption. The discussion also considers the potential future applications of LLMs and how they will be integrated into existing systems.