The Pulse

Stanford’s HomeBody Has GPT Astra Direct a Humanoid in the Kitchen

A Stanford and Caltech team presents HomeBody, which lets GPT Astra direct a Unitree G1 humanoid by calling reusable motor skills. The robot explores a kitchen, builds a digital twin and spatial memory, then tidies up and fetches medicine f

AI.info Team · Written with AI · fact-checked · 2026-09-26

Researchers at Stanford University’s Movement Lab and Caltech have built HomeBody, a system that lets a frontier vision-language model direct a humanoid through household tasks by choosing from a library of reusable motor skills. The project pairs OpenAI’s GPT Astra, running remotely, with a Unitree G1 robot whose local software handles navigation, grasping, placing objects and opening drawers. The model decides what to do and where; the skills plan and execute the physical movements.

A skill library in place of a learned action model

A common design for humanoid autonomy chains three layers: a vision-language model reads the scene and the instruction, a learned vision-language-action model turns that into commands, and a whole-body controller executes them. Gio Huh, a Caltech undergraduate research fellow at the Movement Lab, is the first author, working with Cayden Gu and Takara E. Truong under advisers C. Karen Liu and Guy Tevet. The team questions the middle layer. They wrote on the project page:

“As frontier VLMs such as Astra become more capable, we ask whether a learned VLA is still needed between high-level reasoning and the robot’s skills.”

Gio Huh, Cayden Gu, Takara E. Truong, C. Karen Liu and Guy Tevet, Caltech and Stanford University

HomeBody’s answer is to connect the high-level model straight to the skills, which share one interface for targets and execution results. The model passes its choice through a structured tool call and never generates low-level movement commands itself. New skills, whether a learned policy or a classical algorithm, can be added through the same interface.

Two kitchen tasks, with no kitchen-specific training

The team demonstrates the system cleaning a kitchen by gathering coffee bags onto an island and discarding milk and orange-juice cartons that have gone bad. In a second task, the G1 is asked for medicine it cannot see; it uses stored keyframes to find the drawer, opens it with its right hand, retrieves the medicine and hands it over, then throws out a bad carton with its left. The researchers say HomeBody did this in a previously unseen kitchen without environment-specific training data or additional policy learning.

Exploration gives HomeBody a map to remember

Before receiving a task, the robot explores the room. It collects iPhone video, depth-camera observations, LiDAR scans with SLAM, joint poses and waypoints chosen by the model. GPT Astra then acts as a “Real2Sim” agent, using those data to build a digital twin of the room in NVIDIA Isaac Sim. The robot localizes itself with Super Odometry, and camera observations are stored in the shared map frame so the model can recall where an object was after it leaves view.

At each step the model picks a skill and a target from the current camera view, map context, gripper state, recalled observations and the previous skill’s result. Visual tracking corrects a shifting target as the robot approaches, and the picking skill can make a bounded number of retries if a grasp closes on nothing. When recovery runs out, the skill returns the reason to the model, which can reposition, pick a new target or change its plan.

A laptop, a network link and physical limits

HomeBody runs perception, motion planning and its skills on a single Razer Blade laptop with an RTX 4090 GPU, while GPT Astra runs remotely and sends skill requests over the network. Arm and hand commands run at 250 hertz, and the AMO whole-body control policy, which keeps the robot balanced while its arms reach, updates at 50 hertz.

The project page lists the constraints: reconstructing a room adds setup time and API costs, GPT Astra’s reasoning latency creates pauses between skills, and the robot’s reach, manipulation ability and endurance limit task length, including finger-servo overheating during extended operation. The team’s GitHub repository says the code is coming soon, and the page promises more tasks.