### Enhancing Generative AI Security with Mantle: Amazon Bedrock’s Next-Generation Inference Engine
At Amazon, our culture emphasizes honest and transparent discussions about growth opportunities, allowing us to invest in innovation that raises the standard of value for our customers. Recently, we showcased the effectiveness of this culture through *Mantle*, our cutting-edge inference engine for Amazon Bedrock. As generative AI workloads evolve, it’s essential to optimize how we deliver inferencing to our customers. This led to the development of Mantle, designed with both innovation and security at its core.
### Prioritizing Security in AI Inference
As we reimagined the architecture of our next-generation inference engine, ensuring robust security emerged as our top priority. AWS shares our customers’ unwavering focus on security and data privacy; these values have been central since the inception of Amazon Bedrock. The opportunities presented by generative AI inference workloads are immense, offering customers a chance to leverage the latent value of their data. However, these opportunities come with an equally critical responsibility to maintain the highest standards of security, privacy, and compliance, especially when dealing with sensitive data.
### Operational Security Standards of Amazon Bedrock
Amazon Bedrock is designed with the same operational security protocols that our customers expect from AWS. Utilizing a least privilege model, each AWS operator has access only to the minimal set of systems required for their assigned tasks, with access time limited to when needed. All access to systems that manage customer data is logged, monitored for anomalies, and subjected to audits, ensuring stringent oversight. Furthermore, within Amazon Bedrock, customer data is never used to train models. Model providers do not have any access to customer data, as inferencing occurs exclusively within Amazon Bedrock-owned accounts—an architecture that emphasizes a strong security posture.
### Zero Operator Access with Mantle
With the introduction of Mantle, we’ve taken security to the next level. Inspired by the AWS Nitro System, Mantle is architected with a Zero Operator Access (ZOA) philosophy. This intentional design decision eliminates any technical means for AWS operators to access customer data. Instead, system administration relies on automation and secure APIs that ensure customer data remains protected. No AWS operator can sign in to underlying compute systems or access any customer data—this includes inference prompts or completions. Common interactive tools like Secure Shell (SSH) or AWS Systems Manager Session Manager are not part of the Mantle infrastructure.
### Secure Data Processing with Advanced Attestation
Mantle employs the newly released EC2 instance attestation capabilities, creating a hardened, constrained, and immutable compute environment for processing customer data. Critical services managing model weights and executing inference operations on customer prompts are supported by high assurance cryptographically signed attestation measurements from the Nitro Trusted Platform Module (NitroTPM). When a customer interacts with a Mantle endpoint—such as `bedrock-mantle.[regions].api.aws`—their data, or prompts, leaves their environment encrypted via TLS, secured all the way to the Mantle service, which operates under a ZOA framework.
### Continuous Commitment to Data Protection
Mantle’s design reflects AWS’s long-term commitment to secure and protect our customers’ data. This focus not only motivates teams across AWS to elevate security standards continuously but also allows us to offer foundational confidential computing capabilities, like NitroTPM Attestation, for customer use on Amazon Elastic Compute Cloud (Amazon EC2).
We are dedicated to ongoing investments in enhancing data security and to providing customers with increased transparency and assurance regarding our practices.
### About the Author
**Anthony Liguori** is an AWS VP and Distinguished Engineer for Amazon Bedrock, and the lead engineer for Mantle.
Inspired by: Source
