The Pulse
GPT-6 Astra Tops 40 Robot Policies, but Precision Still Fails
An arXiv study evaluates GPT-6 Astra on 42 RoboDojo manipulation tasks and finds a high aggregate ranking alongside persistent weaknesses in precision, dynamic control and bimanual coordination.
AI.info Team ·
GPT-6 Astra records a 22.48% average success rate across 2,100 robot-manipulation trials, ranking above every public policy included in the RoboDojo comparison. The result gives a large language model a strong showing as a direct robot controller, but the evaluation also finds that Astra performs poorly on tasks requiring precision, dynamic control or complex coordination between both arms.
The findings appear in a paper submitted to arXiv on September 21, 2026. The study examines whether a large language model can act as a robot-manipulation policy without task-specific fine-tuning. In this setup, the model is responsible for producing actions rather than serving only as a high-level planner while another learned policy controls the robot.
“We find that Astra exhibits a sharply polarized capability profile.”
Wenbo Zhang, the paper’s submitting author, is listed among the study’s authors.
Astra leads the benchmark comparison
The researchers evaluate three language models on all 42 RoboDojo tasks and compare their scores with 40 public policies. Astra uses the benchmark’s official 50-episode-per-task protocol, resulting in 2,100 trials. It achieves a 28.97 Score and a 22.48% average success rate, placing it above every public entry in the comparison.
The aggregate result does not mean Astra performs consistently across all forms of manipulation. The paper describes its capabilities as sharply polarized: the model generalizes well on tasks that require semantic understanding but not high-precision control, while struggling when success depends on exact physical execution.
The comparison also includes GPT-5.5 and DeepSeek-Flash. GPT-5.5 reaches a 0.88% average success rate, while DeepSeek-Flash reaches 1.92%. DeepSeek-Flash is evaluated using 10 episodes per task, while Astra and GPT-5.5 use the official 50-episode protocol. All three models use the same post-processing described in the study.
Semantic understanding helps
Astra’s strongest performance appears on tasks where the robot must interpret a scene or instruction and generalize beyond a narrowly specified motor routine. The paper says the model performs well on tasks requiring semantic understanding, including situations in which it must respond to the task’s meaning rather than rely only on precise, repeated movements.
That pattern helps explain why Astra can lead the overall public-policy comparison despite its shortcomings in physical control. A model may identify the relevant object, understand the requested manipulation and select a plausible sequence of actions, yet still fail when the final result depends on exact alignment, timing or force.
Precision and dynamic control remain weak
The study reports poor performance on tasks that require high-precision control, dynamic responses or complex bimanual coordination. These limitations separate Astra’s broad semantic abilities from the reliability needed for demanding physical manipulation.
The paper’s abstract does not present a task-by-task account of those failures, so the aggregate findings do not establish how Astra performs on individual benchmark activities. They do show that the model’s overall ranking is uneven: success on semantically demanding tasks is accompanied by substantial weakness when the robot must maintain precise physical control.
The researchers also examine in-context use of one-shot demonstrations. The experiments show no aggregate benefit from providing a single demonstration of the desired behavior. At the same time, selected interaction traces show the model making corrections within an episode after perturbations or unexpected outcomes. Those traces indicate that such corrections can occur, but the abstract does not claim that demonstrations improve the model’s overall performance.
A strong aggregate result with clear limits
Overall, the evaluation presents GPT-6 Astra as the strongest of the language models tested in this RoboDojo setting and as a policy that ranks above the 40 public entries included in the comparison. Its 22.48% average success rate and 28.97 Score show that a general-purpose language model can produce meaningful manipulation behavior without task-specific fine-tuning.
The result has a defined boundary. Astra’s performance remains unreliable when tasks demand precision, dynamic control or coordinated two-arm behavior. The paper therefore treats the evaluation as initial evidence for the potential of a general-purpose manipulation model, not as evidence that language-model control has solved reliable robot manipulation.