The Pulse
NVIDIA Expands Cosmos With Training and Evaluation Tools
NVIDIA expanded Cosmos with data-generation blueprints, model-training tools and simulation checkpoints for physical AI development. The October 21 announcement also cites partner projects involving autonomous-vehicle data, industrial inspe
AI.info Team · Written with AI · fact-checked ·
NVIDIA’s expanded Cosmos toolkit is designed to let developers test physical-AI systems in generated data and interactive simulations before trying them in the real world. But the new workflow has a boundary: NVIDIA describes deployment through its NIM microservices for supported Cosmos models, even as its training tools target a wider range of open models.
The company announced the additions on October 21 at GTC Berlin, presenting Cosmos as a set of connected tools for preparing data, adapting models and evaluating behavior. Its examples span autonomous driving, industrial inspection and video monitoring, though performance figures in the announcement are company-reported results or projections.
Five blueprints target difficult training data
Cosmos Data Factory now offers five cloud-native blueprints for preparing or generating training data. They cover autonomous-vehicle datasets, image-attribute augmentation, synthetic event videos, defect images for inspection and video augmentation that changes weather or lighting while preserving scene structure. NVIDIA says the Data Factory is available through GitHub, Microsoft Azure, Nebius and CoreWeave.
The update also extends Cosmos Curator, NVIDIA’s GPU-accelerated video-dataset curation tool, to support multimodal, multisensory inputs and processing blocks. The blueprints bring labeling, curation, augmentation and evaluation into a shared toolset, while agent skills can coordinate parts of data processing and synthetic-data generation.
Porsche and ANYbotics show different uses
NVIDIA says Porsche is using Cosmos Data Factory through Voxel51 on Microsoft Azure to develop a shared autonomous-vehicle data workflow across Volkswagen Group brands. The company expects the project to cut repeated physical test drives by up to 90% and shorten development cycles by up to 85%; NVIDIA presents those figures as expected outcomes, not completed measurements.
ANYbotics has generated more than 1,500 synthetic video clips with the blueprints, using Encord to curate real-world data and validate outputs. The material is intended to help the company’s industrial-inspection robots recognize liquid spills and other changes between facility visits. Ultralytics, meanwhile, retrained its YOLO model with real images and targeted synthetic defects; NVIDIA says the work improved defect localization and recall but gives no numerical result.
Training tools reach beyond Cosmos models
Cosmos Framework adds training, optimization and deployment tools and recipes as agent skills. Developers can fine-tune open models, use reinforcement learning, and apply distillation and quantization. NVIDIA names GR00T, Alpamayo, Qwen and Pi0.5 among the models developers can adapt, alongside other compatible models on Hugging Face.
One example comes from MicroAGI, which post-trained Cosmos 3 on videos of people manipulating objects for an egocentric manipulation task. NVIDIA says the robotics research lab distilled a 32-billion-parameter reasoner into an 8-billion-parameter Nano model with no loss in accuracy. The announcement does not specify the evaluation method behind that comparison.
Simulation moves policy tests ahead of deployment
New Cosmos 3 Nano variants and accompanying code support simulations for autonomous-vehicle and bimanual-robot policies. Developers can use repeatable, multiview, multisensor scenarios to train and evaluate policies before real-world testing, including situations that may be difficult to capture outside simulation.
For deployment, NVIDIA says updated Cosmos 3 NIM microservices deliver up to twice the inference speed with no quality loss in validated configurations. Its Metropolis Blueprint for video search and summarization, version 3.3, uses Cosmos NIM and is described as enabling teams to monitor 40% more video streams on the same GPU. Those figures apply to NVIDIA’s stated validated configurations and blueprint claims, not to every model or workload.
The architecture NVIDIA describes is modular: Data Factory prepares datasets, Framework trains or adapts models, and NIM serves supported Cosmos models. The release broadens the tools available for data generation and policy evaluation, while keeping the company’s stated deployment path tied to that supported-model set.