NVIDIA announced a series of new world AI models, libraries, and infrastructure for robot developers at the SIGGRAPH conference on Monday. The most notable among them is Cosmos Reason, a "reasoning" visual language model with 7 billion parameters, specifically designed for physical AI applications and robotics.
Also added to the existing Cosmos series are Cosmos Transfer-2, which can accelerate the generation of synthetic data based on 3D simulation scenarios or spatial control inputs, as well as a lightweight version of Cosmos Transfer that emphasizes speed optimization. NVIDIA stated that these models can be used to generate synthetic datasets of text, images, and videos required for training robots and AI agents.
According to the introduction, Cosmos Reason has memory and physical understanding capabilities, enabling it to infer the next action of an entity agent as a planning model. Applications include data filtering, robot planning, and video analysis.
NVIDIA also released a new neural reconstruction library, which includes a rendering technology that uses sensor data to simulate the real world in 3D. This technology will also be integrated into CARLA, a widely popular open-source simulator platform. Additionally, the Omniverse software development kit has also been updated.
In terms of hardware and platforms, the company introduced a new server designed for robot development workflows — the NVIDIA RTX Pro Blackwell Server, offering a unified architecture support; as well as a cloud-based management platform, NVIDIA DGX Cloud.
This series of releases shows that NVIDIA is accelerating its efforts in the robotics field, aiming to find the next major application scenario for its AI GPUs beyond AI data centers.