The Pulse
Preprint Reports HALO Achieved 91% Success Across 360 Humanoid Trials
A preprint describes HALO, a humanoid autonomy system trained on simulated vision-language-motion data and tested in six real-world home environments. The authors report a 91% success rate across 360 trials without scanning or mapping those
AI.info Team · Written with AI · fact-checked ·
HALO achieved a 91% success rate across 360 trials in six unmapped real-world environments, according to a preprint by its authors. They report that the humanoid autonomy system transferred zero-shot to the physical world, despite not seeing a real home during training.
A data bottleneck for humanoid autonomy
The preprint identifies a shortage of paired data as a constraint on building generalist humanoid systems. Such systems need to map images and free-form language to low-level joint commands, which requires examples connecting visual input and instructions to the movements that carry them out. “Mapping pixels and free-form language to joint commands, however, demands paired supervision: an egocentric image, an instruction, and the precise whole-body trajectory that satisfies it,” the authors write.
The authors contrast their approach with methods that use human teleoperation or wearable capture devices. Producing an hour of data with those methods can require at least an hour of human effort, they write. HALO instead uses a simulation pipeline to generate vision-language-motion data, with data volume tied to computing resources rather than people acting out each episode.
Training across simulated homes
The training corpus contains 300 hours of episodes across a thousand procedurally generated homes. For each episode, a motion-diffusion model generates whole-body movement, a procedural generator supplies an environment, and a simulator places the movement in that environment to render the robot’s sensory inputs. The authors say the homes do not need to be visited or scanned to create the data, and that automated verifiers help maintain data quality.
The system is designed to run onboard the physical robot. Its autonomy stack has three layers. An orchestrator uses an off-the-shelf vision-language model, invoked every two seconds, to turn a free-form request into subtasks that identify a skill and the object it applies to. A motion generator runs at 3 hertz and combines a vision-language-model-based encoder with a diffusion-transformer decoder. From an egocentric RGB image and a subtask, it produces a five-second whole-body motion plan consisting of joint angles and pelvis poses.
A motion tracker runs at 50 hertz. It uses the motion plan together with depth information and proprioception from the robot to execute the movement through joint-level actuator commands. The preprint says the tracker is trained through large-scale reinforcement learning to follow motion plans while respecting balance, contact and actuator limits.
Skills and real-world trials
HALO’s reported skills include sitting on chairs, navigating to named objects, stepping sideways through half-open doors, crouching and crawling beneath tables, and pushing boxes. The system can combine these abilities into longer sequences in response to natural-language requests, according to the authors.
For the physical evaluation, the authors deployed HALO in six real-world home environments that were not seen during training and had not been scanned or mapped. They report a 91% success rate across 360 trials. The preprint describes this as direct transfer from simulation to previously unseen physical environments, with no preparatory scan of the target setting.
Preprint details
“Perceptive Humanoid Motion from Simulation” was submitted to arXiv on Nov. 4, 2026, and is listed as a robotics preprint. Its authors are Siyuan Hu, Abulikemu Abuduweili, Yue Meng, Shipeng Liu, Quentin Leboutet, Chen Chen, Matthias Müller and Vladlen Koltun.