Alibaba Cloud has introduced a new pooling system that reportedly reduces the use of Nvidia GPUs by 82%. This innovative approach aims to optimize cloud resource management and enhance efficiency for users relying on high-performance computing. The initiative reflects Alibaba's efforts to compete in the cloud services market against other major players.

Alibaba's new AI chip is designed to compete directly with NVIDIA’s H200, aiming to capture a share of the growing AI hardware market. The chip boasts advanced capabilities tailored for AI workloads and is positioned to challenge NVIDIA's dominance in the sector. With significant investments in AI technology, Alibaba is poised to leverage its infrastructure to enhance performance and efficiency.

Alibaba Cloud has developed a new pooling system called Aegaeon that significantly reduces the number of Nvidia GPUs required for large language model inference by 82%, allowing 213 GPUs to perform like 1,192. This innovative approach virtualizes GPU access at the token level, enhancing overall output and efficiency during periods of fluctuating demand. The findings, which were published in a peer-reviewed paper, highlight the potential for cloud providers to maximize GPU utilization in constrained markets like China.