As more enterprises transition to leveraging artificial intelligence (AI) for training and inference tasks, the need for robust data and code protection has never been more critical. This is especially true for large language models (LLMs), which often process sensitive data. Many organizations cannot afford to place their data in cloud environments due to the inherent risks associated with data sensitivity. Such data may contain personally identifiable information (PII) or proprietary company information, while the trained models themselves hold significant intellectual property (IP) value.
Enter NVIDIA Confidential Computing (CC), a pioneering solution designed to safeguard large AI models and their underlying data. Launched in 2023, NVIDIA CC allows enterprises to strike the perfect balance between performance and security. NVIDIA has actively collaborated with CPU partners, cloud providers, and independent software vendors (ISVs) to facilitate a seamless transition from traditional, accelerated workloads to confidential, accelerated workloads.
This article will delve into the latest advancements in Secure AI, specifically the Protected PCIe (PPCIE) feature available on NVIDIA HGX H100 and HGX H200 8-GPU systems. Here are the key highlights:
- Protected PCIe (PPCIE) Mode: This innovative mode protects multi-GPU configurations, accommodating eight GPUs and four switches within a Confidential Virtual Machine. It differs from the single GPU setup previously configured using CC mode.
- NV Link Encryption Removed: To enhance performance, NV Link encryption has been eliminated, optimizing data flow between GPUs.
- Attestation Changes: This update allows for the measurement of both GPU and switch configurations operating in PPCIE mode, ensuring that the environment remains secure.
Importance of Security
In today’s interconnected digital landscape, security is paramount. The vast amounts of data generated daily hold immense potential for businesses, influencing the future trajectory of entire industries. For years, various security solutions have focused on protecting data-in-motion, such as during transmission over the Internet, and data-at-rest, such as through encryption of stored information.
However, many vendors overlooked a critical aspect: data in use. Often, data remains exposed in cleartext, making it vulnerable to unauthorized access and manipulation. NVIDIA’s Confidential Computing addresses this significant gap by securing data in use, thereby preventing unauthorized users from accessing or altering sensitive information.
Hardware and Software Security for NVIDIA GPUs
To effectively implement PPCIE on NVIDIA GPUs, specific hardware and software requirements must be met. Let’s explore what you’ll need to get started.
Hardware
Implementing PPCIE necessitates a system equipped with NVIDIA H100 Tensor Core GPUs or NVIDIA H200 Tensor Core GPUs within an HGX 8-GPU setup. Additionally, the CPU must support a trusted execution environment (TEE) to ensure proper operation.
CPU CC Technology
- AMD SEV-SNP
- Intel TDX
Supported CPUs
- AMD Milan (EPYC 7XX3) or AMD Genoa (EPYC 9XX4)
- Intel Emerald Rapids (5th gen Xeon scalable) and Intel Granite Rapids (6th generation Xeon scalable)
Supported GPUs
- All GPU protections and firewalls enabled on the NVIDIA Hopper architecture, including NVIDIA HGX H100 8-GPU 80 GB and HGX H200 NVL systems
Software
- NVIDIA driver: CUDA 12.8 Data Center Driver (r570) or later
- NVIDIA firmware 1.7.0 or later
- Supported hypervisors: Microsoft Azure Hyper-V, KVM
- Supported operating systems
- AMD: Ubuntu 25.04
- Intel: Ubuntu 24.04 with patches
Getting Started
Support for PPCIE is now generally available on CUDA 12.8. This enhancement allows organizations to deploy their LLMs while taking full advantage of 8-GPU performance alongside state-of-the-art hardware security measures.
For comprehensive deployment guidance and additional documentation regarding Secure AI (Protected PCIe), refer to the NVIDIA Deployment Guide. To explore more about NVIDIA’s security offerings, including NVIDIA Confidential Computing, visit NVIDIA Trusted Computing Solutions.
Inspired by: Source
