Azure's ND GB300 v6 virtual machines have set a new record in AI inference, achieving 1,100,000 tokens per second with the Llama2 70B model. This performance surpasses the previous record of 865,000 tokens per second from the ND GB200 v6 by 27%. The ND GB300 v6 machines utilize NVIDIA's Blackwell architecture, which provides 50% more GPU memory and 16% higher Thermal Design Power (TDP) compared to its predecessor. This combination of hardware enhancements positions Azure's offering as a leading option for large-scale AI deployments. The tests were conducted using 18 virtual machines in a single NVIDIA GB300 NVL72 rack, with performance metrics showing an average throughput of around 61,163 tokens per second per node. Notably, the ND GB300 v6 machines deliver five times the throughput per GPU than the previous ND H100 v5 machines. The benchmarks also indicate significant gains in GEMM efficiency, high-bandwidth memory throughput, and faster CPU-to-GPU transfer speeds due to improved NVLink connectivity. For developers looking to replicate these results, the article provides a step-by-step guide on setting up the environment and running benchmarks. Key steps include cloning specific repositories, downloading necessary models and datasets, and configuring Docker containers for the tests. The use of FP4 precision during inference plays a crucial role in achieving high speeds while maintaining accuracy. Overall, this breakthrough highlights Azure's commitment to enhancing AI infrastructure for enterprise needs.