Fireworks.ai: A New Era of Serverless Inference on Hugging Face Hub

In the fast-paced world of artificial intelligence, speed and efficiency are paramount. Fireworks.ai has recently joined the Hugging Face Hub as a supported Inference Provider, transforming the way developers and researchers interact with machine learning models. This article delves into how Fireworks.ai enhances your workflow, making model inference faster and easier than ever.

What is Fireworks.ai?

Fireworks.ai is a robust platform that provides serverless inference capabilities for AI models. This means you can run complex models without needing to manage the underlying infrastructure. With Fireworks.ai, you can seamlessly integrate AI into your applications, allowing for real-time data processing and immediate results.

Key Features of Fireworks.ai

1. Blazing-Fast Inference: Fireworks.ai is designed to deliver ultra-fast inference times, ensuring that you get responses in milliseconds, regardless of the model you’re using.

2. Serverless Architecture: You don’t have to worry about server management. Fireworks.ai handles all the backend complexities, allowing you to focus on building and scaling your applications.

3. Wide Model Support: Fireworks.ai supports a variety of models hosted on the Hugging Face Hub, making it a versatile choice for developers working across different AI domains.

4. Easy Integration: Fireworks.ai is integrated into the entire Hugging Face ecosystem, allowing you to run inference directly on model pages and across various libraries and tools.

How to Use Fireworks.ai

In the Website UI

Using Fireworks.ai is straightforward. Simply navigate to the Hugging Face Hub and search for models supported by Fireworks. The user-friendly interface allows you to quickly find the models you need to implement in your projects.

From the Client SDKs

Fireworks.ai can be accessed via different programming languages, including Python and JavaScript. Here’s how to set it up:

Using Python

To use Fireworks.ai from Python, you’ll need to install the huggingface_hub library. Here’s a quick guide:

More Read

Exploring Google Research Highlights at Google I/O 2024

H Company’s Holo2 Model Sets the Standard in UI Localization Excellence

Unlock the Power of Time-Series Data Using Multimodal Models for Enhanced Insights

Leveraging DeepSomatic AI to Detect Genetic Variants in Tumors

Integrating Hugging Face Models with Amazon Bedrock for Enhanced AI Solutions

pip install git+https://github.com/huggingface/huggingface_hub

Once you’ve installed the library, you can set up the Inference Client as follows:

from huggingface_hub import InferenceClient

client = InferenceClient(
    provider="fireworks-ai",
    api_key="xxxxxxxxxxxxxxxxxxxxxxxx"
)

messages = [
    {
        "role": "user",
        "content": "What is the capital of France?"
    }
]

completion = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1", 
    messages=messages, 
    max_tokens=500
)

print(completion.choices[0].message)

Using JavaScript

For JavaScript developers, Fireworks.ai can be accessed using the @huggingface/inference package. Here’s how to implement it:

import { HfInference } from "@huggingface/inference";

const client = new HfInference("xxxxxxxxxxxxxxxxxxxxxxxx");

const chatCompletion = await client.chatCompletion({
    model: "deepseek-ai/DeepSeek-R1",
    messages: [
        {
            role: "user",
            content: "How to make extremely spicy Mayonnaise?"
        }
    ],
    provider: "fireworks-ai",
    max_tokens: 500
});

console.log(chatCompletion.choices[0].message);

From HTTP Calls

You can also make direct HTTP calls to utilize Fireworks.ai. For example, to call the Llama-3.3-70B-Instruct model using cURL, use the following command:

curl 'https://router.huggingface.co/fireworks-ai/v1/chat/completions' 
-H 'Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxx' 
-H 'Content-Type: application/json' 
--data '{
    "model": "accounts/fireworks/models/llama-v3p3-70b-instruct",
    "messages": [
        {
            "role": "user",
            "content": "What is the meaning of life if you were a dog?"
        }
    ],
    "max_tokens": 500,
    "stream": false
}'

Billing and Pricing

When using Fireworks.ai, billing is straightforward. For direct requests made with a Fireworks key, charges are applied directly to your Fireworks account. If you authenticate through the Hugging Face Hub, you’ll only incur standard Fireworks API rates, with no additional markup.

Important Note: PRO users receive $2 worth of inference credits each month, which can be utilized across various providers. Subscribing to the Hugging Face PRO plan unlocks additional benefits, including ZeroGPU access, Spaces Dev Mode, and significantly higher usage limits.

Light Up Your Projects Today!

With Fireworks.ai now part of the Hugging Face Hub, the possibilities for your AI projects are endless. Experience the ease of serverless inference and accelerate your development workflow. Whether you’re building chatbots, recommendation systems, or any AI-driven application, Fireworks.ai is your go-to solution for efficient and effective model inference.

Explore the full list of models supported by Fireworks.ai and start leveraging this powerful tool today!

Inspired by: Source