PyTorch has released native quantized models, including Phi4-mini-instruct and Qwen3, optimized for both server and mobile platforms using int4 and float8 quantization methods. These models offer efficient inference with minimal accuracy degradation and come with comprehensive recipes for users to apply quantization to their own models. Future updates will include new features and collaborations aimed at enhancing quantization techniques and performance.

ParetoQ is a novel algorithm for low-bit quantization of large language models, unifying binary, ternary, and 2-to-4 bit quantization-aware training. It achieves state-of-the-art performance across all bit widths and offers a reliable framework for comparing quantization methods, demonstrating that lower-bit quantization can surpass traditional 4-bit methods in both accuracy and efficiency. The integration of ParetoQ into the torchao library facilitates easy deployment on edge devices while optimizing accuracy and compression trade-offs.