The article discusses the evolution of GPU architecture, emphasizing the growing disparity between the increasing performance of GPUs and the limited data bandwidth available through traditional buses like PCI Express. It argues for a reevaluation of how data is moved to and from powerful GPUs, highlighting the need for new architectures to address bottlenecks in performance and energy efficiency.

VectorWare is launching as a company focused on developing GPU-native software, aiming to shift the software industry towards utilizing GPUs more effectively as their importance grows in various applications. They emphasize the convergence of CPUs and GPUs and the need for improved tools and abstractions to fully leverage GPU capabilities. With a team of experienced developers and investors, VectorWare is poised to lead this new era of software development.

The article introduces Cuq, a framework that translates Rust's Mid-level Intermediate Representation (MIR) into Coq, aiming to establish formal semantics for Rust GPU kernels compiled to NVIDIA's PTX. It addresses the lack of verified mapping from Rust's compiler IR to PTX while focusing on memory model soundness and offers a prototype for automating this translation and verification process. Future developments may include integrating Rust's ownership and lifetime reasoning into the framework.

Alibaba Cloud has developed a new pooling system called Aegaeon that significantly reduces the number of Nvidia GPUs required for large language model inference by 82%, allowing 213 GPUs to perform like 1,192. This innovative approach virtualizes GPU access at the token level, enhancing overall output and efficiency during periods of fluctuating demand. The findings, which were published in a peer-reviewed paper, highlight the potential for cloud providers to maximize GPU utilization in constrained markets like China.

The article recounts a bug encountered while using PyTorch that caused a training loss plateau, initially attributed to user error but ultimately traced back to a GPU kernel bug on the MPS backend for Apple Silicon. The author details the investigative process which deepened their understanding of PyTorch internals, illustrating the importance of debugging and exploration in mastering the framework. A minimal reproduction script is provided for others interested in the issue.