The Pulse
Skild AI Teaches Robots New Tasks From One Video With NVIDIA
Skild AI says its S1 robot foundation model can learn unfamiliar, multistep tasks from a single video without retraining. The model uses NVIDIA Cosmos, Isaac Lab, Omniverse and TensorRT across data generation, simulation, training and deplo
AI.info Team ·
Skild AI says its S1 robot foundation model can learn unfamiliar, long-horizon tasks from a single video demonstration, while NVIDIA supplies the computing, simulation and data tools behind the system. The claim challenges the standard industrial robotics workflow, in which a new product, layout or process often triggers fresh data collection, retraining and validation.
S1 takes a different route: an operator records the desired task, gives the video to the model as a prompt and lets the robot translate the demonstration into actions. Skild says the model does not update its weights or undergo task-specific post-training before attempting the task.
The companies disclosed the collaboration in a September 10 NVIDIA Blog post. Skild separately published its technical account of S1 in August, describing the system as an in-context learner for robotic manipulation.
Skild’s One-Video Claim Meets Factory Constraints
Industrial robots typically perform narrow, repeatable jobs. Changing the component, moving a workstation or altering the assembly sequence can require engineers to collect new examples and tune a specialist policy. Skild argues that S1 can absorb those changes from a demonstration rather than a new training run.
The company says S1 handles unfamiliar tasks lasting as long as 10 minutes and involving dozens of manipulation steps. Its examples include potting a plant, making a pancake, brewing pour-over coffee and assembling a kit. Those tasks require the system to combine skills in sequences that did not appear together in its training data, according to Skild.
“Learning by experience, and not preprogramming, is the step change that has happened in robotics,” Deepak Pathak, cofounder and chief executive of Skild AI, says in NVIDIA’s post. “NVIDIA Isaac Lab and NVIDIA Cosmos technologies help Skild create the scalable, diverse experience its robots need to learn across many scenarios and embodiments.”
66% Success on Unseen Multistep Tasks
Skild reports a 66% per-step success rate for S1 on new multistep tasks, compared with 9% for a similar language-prompted AI system. The comparison comes from the company’s internal evaluation and represents more than a sevenfold difference, but it is not an independently audited benchmark.
Skild’s own technical write-up says the test tasks ran for four to eight minutes and were drawn from a set of activities outside the model’s pretraining distribution. The company evaluated cumulative per-step success and used human intervention to recover from failures during rollouts, primarily because the comparison system otherwise could not complete the longest unseen tasks.
The model’s strongest reported advantage appears on tasks it has not seen before. Skild says language-prompted systems reached a 9% success rate after training on 100,000 hours of data, while S1 reached 66% using video-based in-context learning on the same scale of evaluation. On tasks within the training distribution, Skild reports that S1 reached roughly 96% accuracy.
A separate comparison estimates that one video demonstration produces performance similar to roughly 380 post-training episodes. Skild says collecting those episodes through teleoperation would require 50 to 100 hours for long-horizon tasks. The estimate comes from interpolating the company’s internal results, not from a production deployment study.
From a Plant-Potting Demo to Blackwell Assembly
Skild says the fastest demonstration-to-execution test involved plant potting. The team recorded one egocentric human video at 9:22 p.m., and S1 began autonomous execution on hardware at 9:27 p.m. The company’s published timeline places the full interval from recording the demonstration to robot execution at 11 minutes, including scene preparation.
The model also handled changes during execution, according to the companies. Skild says S1 can respond when objects move, recover after errors and use substitute objects with similar functions. Its technical post describes examples in which the robot changes how it waters a plant or fills a nearly full glass instead of copying the demonstration mechanically.
The commercial target is less theatrical than pancake flipping. Skild, NVIDIA and Foxconn are deploying the Skild Brain on dual-arm manipulators for high-precision assembly of NVIDIA Blackwell systems. One demonstrated workflow requires a robot to install a busbar and limit block, fasten 16 screws and respond when the scene differs from the original plan.
Skild also says it is working toward deploying S1 at Sumitomo Wiring Systems for wire-harness manufacturing processes that the company describes as previously difficult to automate. Its September 10 business update says Skild has more than 60 paying customers and crossed a $100 million annual recurring revenue run rate 10 months after its first commercial deployment.
NVIDIA Supplies the Training and Simulation Stack
NVIDIA’s role spans more than the GPUs used to train S1. The companies say Skild uses NVIDIA Cosmos models to diversify training data and convert video into structured descriptions, with Cosmos Curator helping organize and filter data.
Skild also uses NVIDIA Omniverse libraries and Isaac Sim to create physically based virtual environments, test unusual conditions and validate robot behavior before deployment. Isaac Lab provides the reinforcement-learning framework, while the Newton physics engine models forces, contact, collisions and pressure in an effort to narrow the gap between simulation and hardware.
The two companies are jointly developing GPU-accelerated simulation solvers for robot contact, gripping and object manipulation. NVIDIA says those solvers will eventually become available to other developers through Newton. Nsight tools help identify training bottlenecks, and TensorRT is used to optimize inference for real-time robot responses.
Deployment Is the Test S1 Still Has to Pass
Skild’s own account places deployment at the center of its evaluation strategy. The company warns that a successful robot video can conceal whether a system works 5%, 10% or 99% of the time, because teams can repeat a demonstration until they capture a good clip.
That distinction matters for S1. A single-video prompt may reduce the time needed to configure a new task, but factories still require predictable cycle times, safe recovery behavior and reliability across equipment, materials and operators. Skild says its commercial deployments are intended to feed experience back into the general model where customer agreements allow it.
The immediate evidence is a mix of internal benchmark results and early industrial deployments, not a public record of sustained production performance across the company’s customer base. For now, S1’s clearest proposition is specific: show the robot a new multistep task, avoid a new fine-tuning run and move from demonstration to hardware in minutes rather than weeks of task-specific preparation.