Nvidia has released new physical AI research tools, agent workflows, and open source models to train more advanced AI systems for the real world.
Unveiled this week at the Computer Vision and Pattern Recognition conference in Denver, the update builds on Nvidia’s recently launched update. Cosmos 3 World Foundation Model and is designed to help researchers automate key stages of physical AI development, including simulation, synthetic data production, policy training, and evaluation.
Physical AI refers to AI systems that interact and operate with the physical world, including self-driving vehicles. industrial robot and embedded AI agents.
The company said the new capabilities address a major challenge facing engineers in the industry: creating scalable workflows to train and test AI before deployment in the real world.
“The main challenge in physical AI research is not just developing robust models. It is building a complete workflow around them,” Nvidia said in a statement. blog post. “Today, these steps are fragmented across separate instruments, slowing experiments as researchers struggle to link them together.”
agent skills
New agent skills have been integrated into announcements nvidia omniverseIsaacSim, IsaacLab, and Cosmos enable developers to automate tasks such as scene reconstruction, simulation setup, environment creation, and reinforcement learning workflows.
For autonomous vehicle development, Nvidia introduced tools to help researchers address the industry’s “long-tail problem” – difficult-to-capture driving scenarios that are critical for training and validation.
To bridge this gap, Nvidia said its AI agents can now automate the reconstruction of real-world driving environments from fleet data and generate synthetic edge-case scenarios for testing.
The AI ​​giant also introduced Alpamayo 2 Super, a 32 billion-parameter vision-language-action model. autonomous driving. The system is designed with advanced logic capabilities, enabling it to function autonomously across the full driving stack.
In the Vision AI area, Nvidia expanded its video analytics capabilities with an update to its Metropolis platform, including tools for video search, summarization, and synthetic data generation.
The company said these capabilities will help developers create AI agents capable of understanding complex scenes, identifying events, and generating alerts from video streams.
Robotics was another major focus, with new agent skills designed to automate simulation and training workflows. This reduces the manual labor typically required to create virtual environments and train robots within them.
Release highlights Nvidia’s growing focus physical ai As a major development area. With the update, the company is positioning virtual environments as a key tool for developing AI systems that can operate safely in the physical world.
Nvidia’s new Physical AI suite is now available via GitHub, while its synthetic data generation tools (neural reconstruction, video augmentation, and defect image generation) are available on Nvidia Brave with free trial credits for researchers.
