Kubernetes: The Foundation for the Future of AI
Kubernetes, the open-source system recognized for efficiently managing how applications run across vast networks of AI factories, is steadily becoming the backbone of artificial intelligence infrastructure. As AI continues to advance, the demands on Kubernetes are escalating, pushing the boundaries of cloud-native capabilities.
At the upcoming KubeCon + CloudNativeCon, NVIDIA is making significant strides to meet these growing demands by deepening its commitment to open-source solutions. This flagship conference is where leaders from the Kubernetes and cloud-native communities converge to discuss innovations and strategies for optimizing cloud infrastructures.
NVIDIA’s Contributions to Open Source
In an exciting development, NVIDIA is contributing NVIDIA OpenShell, an open-source runtime designed to secure and govern autonomous agents, to the Cloud Native Computing Foundation (CNCF). As a part of the CNCF Sandbox initiative, OpenShell is set to be governed collaboratively, bringing together diverse organizations and harnessing the expertise of global security professionals.
In addition to OpenShell, NVIDIA is also supporting the Agent Substrate, an open-source framework initiated by Google. This project, now accepted into the CNCF Sandbox, is aimed at operating long-running agent workloads efficiently at scale on Kubernetes.
Enhancements in NVIDIA’s DSX AI Factory Platform
NVIDIA is also unveiling new components of its DSX AI Factory Platform: the NVIDIA Kubernetes Engine (NVKE), NVIDIA Kubernetes Recipes (NVKR), and NVIDIA Snapshot. These tools are specially designed to assist teams in deploying, managing, and scaling GPU infrastructure on Kubernetes, creating a robust ecosystem for AI factories.
More Read
These advancements signify a substantial leap towards making Kubernetes the premier choice for running AI workloads at scale. They empower developers to construct safer and more efficient AI infrastructures, paving the way for future innovations.
Securing AI Agents in the Open
NVIDIA’s introduction of OpenShell to CNCF addresses a critical challenge faced by the cloud-native community: ensuring that agents operate within secure environments. According to Jim Zemlin, CEO of the Linux Foundation, the OpenShell initiative offers developers a collective foundation to build upon, thus allowing for collaborative development that enhances security for the next generation of cloud-native applications.
OpenShell enables organizations to maintain control over AI agents, utilizing deterministic controls to enforce policies as agents navigate through various tasks, such as accessing inference services and executing workloads. Its extensible architecture facilitates integration with existing enterprise systems, enabling improved governance across diverse deployment scenarios.
In less than a year since its launch, OpenShell has gained traction, with hundreds of organizations actively utilizing the framework, leading to tens of thousands of contributions.
Scaling Agent Execution on Kubernetes
Scaling agent workloads is another critical issue that organizations face. NVIDIA’s contribution to Agent Substrate addresses this by consolidating agent execution, inference serving, and resource scheduling on Kubernetes. The project preserves agent environments between tasks, allowing for efficient scaling without the need to maintain dedicated compute resources for each session.
NVIDIA is also working closely with Google on integrating OpenShell with Agent Substrate, enhancing the ability of enterprises to secure and manage their agents across various cloud and on-premises systems.
Building an Open Foundation for AI Factories
For AI factories, providing managed Kubernetes services while retaining control over infrastructure and customer experiences is paramount. The public preview of NVKE and NVKR offers open-source solutions that simplify the deployment of managed Kubernetes services on GPU infrastructures, automating essential tasks like cluster lifecycle management and ensuring secure, version-controlled images.
The ecosystem surrounding NVKE and NVKR is expanding rapidly, with numerous major players—such as Mirantis and OpenNebula Systems—aligning with NVIDIA to adopt these capabilities.
Accelerating Inference with NVIDIA Snapshot
To address the bottlenecks associated with launching additional inference capacity, NVIDIA has introduced NVIDIA Snapshot. This modular component significantly reduces the time required to load models and activate workloads. With Snapshot, AI workloads can be restored quickly, supporting enhanced startup times and recovery for agent workloads on Kubernetes.
Several design partners, including Crusoe and Saturn Cloud, have successfully tested Snapshot on their systems, achieving remarkable reductions in workload readiness times. These enhancements underscore the potential of Snapshot in optimizing scaling and recovery processes for AI applications.
By attending KubeCon, NVIDIA aims to further engage with the community and contribute to the evolution of Kubernetes as a foundational platform for AI workloads on a global scale. With its open-source initiatives, NVIDIA is committed to fostering collaboration and innovation in the realm of AI and cloud-native technologies.
Inspired by: Source
