Organizations are increasingly adopting AI and machine learning, often requiring these capabilities at distributed locations. This demand is driven by needs like local data processing for compliance and low-latency inference requests. To address these challenges, AWS and NVIDIA have integrated GPU resource management with Amazon Elastic Kubernetes Service (EKS) to facilitate efficient workloads across various environments, including on-premises setups and edge locations. The collaboration introduces native support for NVIDIA Run:ai across AWS Local Zones, AWS Outposts, and EKS Hybrid Nodes. This allows users to manage GPU resources flexibly, optimizing performance regardless of whether workloads run in AWS Regions, on-premises, or at the edge. The architecture enables high availability and disaster recovery strategies while adhering to local data residency rules. Organizations can leverage the same APIs and management tools across all deployments, simplifying operations and maintaining consistent resource management practices. When deploying AI workloads at the edge, teams face specific challenges. For training, considerations include optimizing workload distribution and ensuring secure, efficient connections between on-premises and cloud GPU clusters. Inference deployments require maintaining consistent performance across distributed endpoints and managing multiple requests effectively. The new Run:ai capabilities help organizations navigate these issues, improving GPU utilization rates from around 25% to over 75%. Features like dynamic GPU fractionalization and advanced scheduling are designed to enhance efficiency in resource allocation, which is critical in real-time applications like autonomous systems and healthcare diagnostics.