The Pulse
Ai2 Introduces ToyBlocks, a Simulated Tabletop World for Studying Robot Learning
Ai2 introduced ToyBlocks, a simulated tabletop world for training and evaluating robot systems. It says the framework generates demonstrations and tests how models handle tasks such as flipping, sorting and stacking blocks.
AI.info Team · Written with AI · fact-checked ·
Ai2 introduced ToyBlocks on November 6, 2026, as its first FoundationWorld, a controlled simulation framework for studying robot learning. The simulated setting centres on a robot arm and blocks on a tabletop, where tasks include pushing, flipping and stacking. Ai2 says ToyBlocks automatically generates demonstrations and measures robot behaviour over time.
A tabletop world built for repeatable tests
The FoundationWorld approach combines a simulated environment with tools for generating tasks and training demonstrations, along with metrics for tracking performance. Ai2 describes the ToyBlocks setting as deliberately simple, and says researchers can vary training data or model design and repeat tests in the same setting. The blocks’ pictures and letters make it possible to create search, sorting and arranging tasks, including ones that require turning a block to reveal a hidden face.
ToyBlocks varies block size, table height and other physical properties from scene to scene to test how models handle changes in geometry. The system also supports tests on tasks held out of training, letting researchers compare what models learn from different mixtures of demonstrations.
Demonstrations generated at up to 24 times real time
ToyBlocks uses reinforcement learning and motion planning to generate skills for short sequences of actions. Skill training takes place in Isaac Lab, Nvidia’s open-source framework for robot learning in simulation. The learned skills include pushing a block across a table or flipping it without picking it up.
A task-and-motion planner selects and sequences skills, then plans the robot’s movements between them. ToyBlocks can generate and execute hundreds of plans in parallel on a single graphics processing unit, reaching up to 24 times real-time performance. The resulting demonstrations include labels identifying the task or skill being performed.
More than a success rate
For evaluation, ToyBlocks reports metrics beyond task success rates. Those metrics include execution speeds, forces exerted on objects and task progress measured using the planner’s specifications. The framework also supports experiments that separate high-level planning from low-level execution, helping researchers identify which part failed on a task.
In one planning test, a system using Claude Opus 5.5 solved all 30 of a group of block-stacking tasks from text descriptions, and also solved 30 out of 30 while observing camera images and choosing actions in a closed loop. In a separate search-and-spell experiment, the agent had to find letter blocks among up to eight blocks, with 50% of the scenes missing a required letter. In this experiment, the agent successfully solved or correctly declared infeasible 63% of the tasks.
Testing agents that learn their own skills
Errors in the search-and-spell test were mostly caused by the agent confusing letters with look-alike letters (13%) or choosing an incorrect orientation for a letter in a stack (15%). The experiments also tested whether an agent could investigate unexpected skill behaviour, run a small test and revise its understanding.
In that experiment, the agent ran its own experiments and wrote its own skill descriptions based on what it found. Those learned descriptions raised success rates from 24% with the default descriptions to 63%, outperforming the 36% achieved with descriptions written by a human expert.