The Pulse
SEAR Tests Whether Robot Agents Learn From Their Own Experience
SEAR measures whether robot agents improve by retaining useful experience or simply by trying again. The benchmark tests seven language models across 330 simulated tasks and five physical robot tasks, finding that shared skills can help, hu
AI.info Team · Written with AI ·
A reusable skill library lifted Gemini-3.6-flash by 46.6 percentage points on simulated tasks with hidden hardware faults, but the same kind of stored experience lowered GPT-5.6-sol’s RoboLab score from 46.9% to 42.4%. The contrast captures the central finding in SEAR, a new benchmark that tests whether robot agents improve by retaining useful experience—or merely by making more attempts.
The paper, submitted to arXiv on October 13, 2026, studies seven language models across 330 simulated tasks and five physical tasks. Its authors write: “Harder tasks make longer sessions worthwhile; on easier tasks, re-rolling short attempts does just as well.” The result is a qualified account of machine learning: agents can carry useful skills forward, but success depends on the model, the task and what gets saved.
“Harder tasks make longer sessions worthwhile; on easier tasks, re-rolling short attempts does just as well.”
Xinyi Wang, Bangzheng Li, Hanyu Wang, Lin Zhang, Andrea Yaoyun Cui, Zhenggang Tang, Shyamal Buch and Dejia Xu, “SEAR: Self-Evolving Agentic Robotics”
SEAR separates learning from another attempt
A rising success rate alone cannot show that an agent has learned. More attempts, a longer work session, access to task information or a lucky run can all make performance look better. “A rising success rate, however, is not proof that the agent learned,” the authors write.
SEAR compares a long work session with three shorter attempts sharing the same time budget. It also compares agents that inherit a library of skills and written memories with agents that start each task without one. A separate judge scores frozen policies on held-out test states, while repeated runs help reveal the role of luck.
The benchmark follows agents across five levels of distance from practice: new starting states, changed conditions, new tasks, a different simulator and, finally, a physical robot. That structure lets the researchers ask not only whether performance improves, but where that improvement still holds.
Stored skills help some models—and hurt others
The library’s effect varies with both model and task. On RoboLab, the benchmark’s hardest simulated family, Kimi-k3 rose from 30.1% without a library to 51.9% with one. GPT-5.6-sol, by contrast, fell from 46.9% to 42.4% with its own library, but scored 60.6% when given another model’s library.
The authors report that stored experience helps most when a model already succeeds sometimes but not reliably. A weaker agent may have little useful code to preserve; a stronger one may have less room to improve. Memories can also preserve errors: the paper describes an agent saving a mistaken measurement as fact, which later tasks then inherited.
Time allocation matters too. A long session can outperform several short restarts when an agent’s first success arrives only after a short attempt would have ended. When a model reaches a successful policy early, splitting the same budget among fresh runs can be just as effective or better.
Practice does not guarantee success on a real robot
SEAR tests three ways for agents to control robots: writing code that becomes the policy, acting live in the control loop, or collecting rollouts and fine-tuning a vision-language-action model. In simulation, shared control and grasp routines transfer across tasks and simulators more reliably than task-specific solutions, the paper reports.
On physical robots, the gap between practice and evaluation scenes was less forgiving. Agents practiced on tabletop tasks using two YAM-Ultra arms, then faced scenes staged by an operator. A streak of ten practice successes did not guarantee success on those new scenes, the authors found.
The hardware results also come with limits the authors spell out: each physical agent had one practice run, and frozen policies were evaluated once on each of ten operator-staged scenes. The physical runs started without a library inherited from simulation, so they test learning on hardware rather than transfer from simulated robots to real ones. The study therefore offers a test framework and evidence of conditional gains—not a blanket demonstration that robot agents can reliably learn from experience.