Deploying Open Models on Amazon Bedrock: A Guide to Leveraging Hugging Face Models

The world of Generative AI is rapidly evolving, and with it comes exciting opportunities for developers to create innovative applications. One of the latest advancements is the integration of popular open models from Hugging Face into Amazon Bedrock through the newly launched Bedrock Marketplace. This groundbreaking feature allows AWS customers to deploy a staggering 83 open models, simplifying the process of building Generative AI applications.

Contents

Understanding Amazon Bedrock and SageMaker Jumpstart
Step-by-Step Guide to Deploying Google Gemma 2 27B Instruct

Deploying the Model
Using the Model with Amazon Bedrock APIs
Cleaning Up Resources

Understanding Amazon Bedrock and SageMaker Jumpstart

Amazon Bedrock provides a fully managed infrastructure designed to support the deployment and scaling of AI models, while SageMaker Jumpstart streamlines the process of model deployment. The Bedrock Marketplace combines the ease of SageMaker Jumpstart with the robust capabilities of Amazon Bedrock, ensuring compatibility with high-level APIs such as Agents, Knowledge Bases, Guardrails, and Model Evaluations.

When you register your SageMaker Jumpstart endpoints in Amazon Bedrock, you will only incur costs for the SageMaker compute resources, while standard Amazon Bedrock API prices apply. This model makes it easier and more cost-effective for developers to experiment with and implement advanced AI solutions.

Step-by-Step Guide to Deploying Google Gemma 2 27B Instruct

Deploying the Model

You have two primary methods to deploy an open model for use with Amazon Bedrock:

Via the Bedrock Model Catalog: This is a straightforward way to access and deploy models directly from the catalog.
Using Amazon Jumpstart: This method allows you to deploy a model and register it with Bedrock for further integration.

In this guide, we will focus on deploying from the Bedrock Model Catalog.

To begin, navigate to the Amazon Bedrock console and ensure you are in one of the 14 supported regions. In the navigation pane, select “Model Catalog” under the Foundation Models section. Here, you can search for serverless models and those available in the Bedrock Marketplace. Filter the results by the Hugging Face provider to discover the 83 open models at your disposal.

For this tutorial, let’s search for and select Google Gemma 2 27B Instruct.

Once you’ve chosen the model, you’ll be directed to the model detail page, which provides insights about the model, its highlights, and sample API calls. To initiate the deployment, click on the Deploy button located at the top right corner.

You will then reach the deployment page, where you can specify the endpoint name, instance configuration, and advanced settings related to networking and service roles for SageMaker deployment. For simplicity, we recommend using the default advanced settings and the suggested instance type.

Before proceeding, make sure to accept the End User License Agreement from the model provider. Once you’re ready, click on Deploy.

Your deployment of the Google Gemma 2 27B Instruct model on an ml.g5.48xlarge instance has been launched, hosted in your Amazon SageMaker tenancy and compatible with Amazon Bedrock APIs. The deployment may take several minutes to complete, and you can track its progress on the Marketplace Deployments page found in the navigation pane.

Using the Model with Amazon Bedrock APIs

Once your model is deployed, you can test it in the Playground through the user interface. However, to invoke the deployed model programmatically using Amazon Bedrock APIs, you need to obtain the endpoint ARN.

In the list of managed deployments, select your model deployment to copy its endpoint ARN.

With the endpoint ARN in hand, you can query your endpoint using the AWS SDK in your preferred programming language or through the AWS CLI. Below is an example of how to use the Bedrock Converse API with the AWS SDK for Python (Boto3):

import boto3

bedrock_runtime = boto3.client("bedrock-runtime")

endpoint_arn = "arn:aws:sagemaker:<region>:<accountid>:endpoint/<endpoint_name>"

inference_config = {
    "maxTokens": 256,
    "temperature": 0.1,
    "topP": 0.999,
}

additional_model_fields = {
    "parameters": {
        "repetition_penalty": 0.9,
        "top_k": 250,
        "do_sample": True
    }
}

response = bedrock_runtime.converse(
    modelId=endpoint_arn,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "text": "What is Amazon doing in the field of generative AI?",
                },
            ]
        },
    ],
    inferenceConfig=inference_config,
    additionalModelRequestFields=additional_model_fields,
)

print(response["output"]["message"]["content"][0]["text"])

This simple code snippet allows you to interact with your deployed model and receive responses based on the input provided. For instance, querying about Amazon’s initiatives in Generative AI can yield insightful responses, showcasing the capabilities of the model.

Cleaning Up Resources

Once you’ve finished experimenting with your model, it’s crucial to clean up and delete your endpoint to avoid incurring unnecessary costs. In the top right corner of the page where you retrieved the endpoint ARN, you will find an option to Delete your endpoint. Click on it to ensure that you stop all resource usage related to the deployment.

By following these steps, you can effectively deploy and utilize open models from Hugging Face on Amazon Bedrock, unlocking new possibilities for your Generative AI applications.

Inspired by: Source

Integrating Hugging Face Models with Amazon Bedrock for Enhanced AI Solutions

Deploying Open Models on Amazon Bedrock: A Guide to Leveraging Hugging Face Models

Understanding Amazon Bedrock and SageMaker Jumpstart