Introducing NVIDIA Isaac GR00T N1.7: Revolutionizing Humanoid Robotics
NVIDIA has released the highly anticipated Isaac GR00T N1.7 — a groundbreaking open-source, commercially licensed Vision-Language-Action (VLA) model tailored specifically for humanoid robots. This model is built on the principle that human-generated data is the most scalable source of intelligence for robotic systems, making it a game changer in the pursuit of advancing robotics capabilities.
TL;DR
- 🤖 GR00T N1.7 is now available for access on Hugging Face and GitHub, providing a robust foundation for humanoid robots.
- 🏭 Designed for factory-floor readiness, with commercial licensing facilitating immediate production deployments in material handling, packaging, and inspection tasks.
- 🧠 Incorporates multi-step reasoning for improved reliability during complex workflows.
- 🖐 Offers expanded dexterous manipulation, allowing for intricate tasks like handling fragile components through finger-level control.
- 🔬 Introduces the first-ever dexterity scaling law: the model demonstrates that more human data dramatically enhances robot dexterity, eliminating the need for extensive teleoperation.
- 🚀 Resources: Available on GitHub and Hugging Face.
What is GR00T N1.7?
The GR00T N1.7 model boasts an impressive architecture comprising 3 billion parameters focused on translating visual observations and natural language instructions into actionable movements for robots. At its core is the Action Cascade architecture that differentiates between high-level reasoning and low-level motor control:
-
System 2 (Vision-Language Model): This component processes image tokens and language inputs to output high-level action tokens. This is where task decomposition and multi-step reasoning take place, essential for executing complex tasks efficiently.
-
System 1 (Diffusion Transformer): A sophisticated 32-layer model that takes the outputs from System 2, along with real-time robot state data, and transforms them into precise motor commands.
This dual-system architecture ensures that robots operate smoothly and accurately across a variety of tasks.
Inputs and Outputs
-
Inputs:
- RGB image frames (at any resolution)
- Language instructions
- Robot proprioceptive state (joint positions, velocities, and end-effector poses)
-
Outputs:
- Continuous-value action vectors corresponding to the robot’s degrees of freedom.
GR00T N1.7 has been validated in diverse applications, including loco-manipulation, tabletop tasks, and dexterous bi-manual operations, utilizing platforms like Unitree G1, YAM manipulator, and AGIBot Genie 1.
Training on Human EgoCentric Video Data
The backbone of GR00T N1.7 is its training on over 20,854 hours of human egocentric video data across various domains such as manufacturing, retail, and healthcare. This training paradigm represents a significant advancement from previous models that depended on limited teleoperation data.
More Read
Innovations from EgoScale
The key insight from this research is the discovery of a scaling law for robot dexterity. More human-centric data directly correlates with enhanced dexterous manipulation. For instance, increasing training data from 1,000 to 20,000 hours has shown to more than double the average task completion rates. This means that robots leveraging GR00T N1.7 can now perform complex interactions with objects that historically posed challenges to generalist robotic models.
Inference & Deployment
NVIDIA makes it easy for developers to implement GR00T N1.7 in their robotic platforms. Here’s how to get started:
bash
git clone –recurse-submodules https://github.com/NVIDIA/Isaac-GR00T
cd Isaac-GR00T
bash scripts/deployment/dgpu/install_deps.sh
source .venv/bin/activate
Then, initiate the policy server with:
bash
uv run python gr00t/eval/run_gr00t_server.py
–embodiment-tag GR1
–model-path nvidia/GR00T-N1.7
This sets the foundation for querying the model to integrate it with various environments.
Performance Metrics
For inference performance, details are provided within the deployment documentation. GR00T N1.7 is compatible with NVIDIA’s latest hardware advancements, including the Ampere, Hopper, Lovelace, and Blackwell architectures, amplifying its utility across diverse robotics applications.
Fine-Tuning on Your Robot
One of the standout features of N1.7 is its ability to be fine-tuned on custom robotic embodiments. This is accomplished using the LeRobot dataset format. Pre-registered embodiments include popular models like UNITREE_G1, LIBERO_PANDA, and OXE_WIDOWX.
Here is how you can initiate fine-tuning:
bash
CUDA_VISIBLE_DEVICES=0 uv run python gr00t/experiment/launch_finetune.py
–base-model-path nvidia/GR00T-N1.7
–dataset-path
–embodiment-tag
–modality-config-path
–num-gpus 1
–output-dir
–max-steps 2000
–global-batch-size 32
For users upgrading from N1.6, the upgrade process is seamless. The only requirement is to point to the nvidia/GR00T-N1.7 model, ensuring compatibility with existing configurations and workflows.
These advancements underscore the capabilities of GR00T N1.7, positioning it as a pivotal tool in the evolution of humanoid robotics. From enhanced dexterity to commercial deployment, GR00T N1.7 sets a new benchmark in the integration of AI with mechanical systems, pushing the frontiers of what humanoid robots can achieve. If you’re building something innovative with GR00T N1.7, the NVIDIA community is excited to hear from you and learn how you’re leveraging this cutting-edge technology in practical applications!
Inspired by: Source
