Nvidia on Monday introduced Cosmos 3, an open world model designed to help robots, autonomous vehicles and other physical systems better understand, simulate and navigate real-world environments.
The model is designed to generate and predict actions in physical settings rather than simply creating images or videos. Nvidia said Cosmos 3 was trained on 20 trillion tokens of multimodal data, including nearly 1 billion images, 400 million real and synthetic videos, ambient audio, text and action data collected from humans and robots. According to Axios, the inclusion of action data enables the model to simulate how machines move and interact with the physical world rather than only modeling visual scenes.
