NVIDIA Releases Cosmos Policy for Robots

NVIDIA's Cosmos Policy is built upon the Cosmos Predict-2 world foundation model (WFM), fine-tuned for robot manipulation tasks. This allows robots to understand temporal structure and physical interactions, simplifying training and deployment. The policy directly encodes robot actions and future states into the model. Cosmos Policy achieves state-of-the-art performance on benchmarks like LIBERO and RoboCasa. In testing, it reached a 98.5% success rate on the LIBERO simulation. This is achieved without significant architectural changes or multi-stage training. The breakthrough lies in representing data; instead of separate networks for perception and control, robot actions, physical states, and success scores are encoded as latent frames, similar to video frames. This allows the model to leverage its pre-existing understanding of physics and how scenes evolve. This approach unifies perception, control, and evaluation within a single temporal latent space. NVIDIA's Cosmos models are part of its broader Physical AI initiative, aimed at training AI on vast amounts of video to understand generic physical dynamics. This includes how objects move, collide, and persist over time. The NVIDIA Robotics Platform offers a hardware and software ecosystem for building AI-powered autonomous robots, combining Jetson embedded computers with Isaac software frameworks. NVIDIA's Isaac platform is used by many top robotics companies. The Isaac platform includes Isaac Sim, a simulation environment built on NVIDIA Omniverse, and Isaac ROS (Robot Operating System). These tools help developers create, test, and deploy AI robots faster. Embodied AI, the integration of AI into physical systems, is a key trend in robotics. It enables robots to interact with the physical world, learn from experience, and adapt to new situations. This differs from traditional AI, which operates in abstract digital environments. NVIDIA is also developing Isaac GR00T, a research initiative and open development platform for general-purpose robot foundation models, particularly for humanoid robots. These advancements are intended to make robots more intelligent and useful in real-world applications.