MobileVLA-R1 2.0: RL-Enhanced Reasoning for Mobile Robot Control

AI summary

Overview

• Research area: Embodied AI / robotics — vision-language-action (VLA) models for mobile robot control, combining multimodal perception, Chain-of-Thought reasoning, supervised fine-tuning, and reinforcement learning (GRPO).
• Technical level: Advanced. The paper assumes familiarity with VLA architectures, Chain-of-Thought supervision, LoRA fine-tuning, cross-attention decoders, and policy-gradient optimization.
• Scope (1 sentence): The paper introduces MobileVLA-R1 2.0, an RL-enhanced VLA framework that explicitly converts structured embodied reasoning into task-level locomotion and behavior commands for navigation, quadruped control, and humanoid mobile manipulation.

What This Paper Is About

Grounding natural-language instructions into reliable robot actions is hard because there is a persistent gap between high-level semantic reasoning and low-level locomotion and manipulation control. Existing VLA systems tend to either reason implicitly or predict actions monolithically, which makes coherent long-horizon decision making and precise, adaptable control difficult to achieve at the same time. The goal of this work is to build an explicit, learnable interface between deliberative reasoning and executable robot behavior that works across heterogeneous robot bodies without retraining the policy per embodiment.

Key Contributions

1. An RL-enhanced VLA framework with an explicit reasoning-to-action interface. MobileVLA-R1 2.0 combines multi-granularity embodied reasoning, supervised Chain-of-Thought (CoT) alignment, and reinforcement learning, following a System-2-to-System-1-to-System-0 paradigm (deliberative reasoning, then task-level action generation, then embodiment-specific low-level control).
2. A reasoning-conditioned action decoder. Instead of parsing control commands out of generated text with hand-designed rules (as in the conference version), the decoder uses two learnable action queries (locomotion and behavior) with cross-attention over observation and reasoning representations, regressing continuous planar velocities (Vx, Vy, ω) and classifying a discrete task-level behavior primitive α.
3. Embodiment-decoupled task-level action interface. The policy predicts semantic task-level actions rather than morphology-specific joint commands; robot-specific low-level controllers realize those predictions, so the same learned policy can be evaluated on different platforms without modification.
4. Broad evaluation including humanoid transfer. Experiments cover VLN-CE (R2R-CE and RxR-CE), QUARD quadruped control, real-world Unitree Go2 deployment, and real-world Unitree G1 humanoid mobile manipulation — with no G1-specific trajectories, demonstrations, task annotations, or policy fine-tuning used anywhere in training.

Main Findings

• VLN-CE improvement: MobileVLA-R1 2.0 achieves an average 1.6 point improvement in Success Rate (SR) on VLN-CE over MobileVLA-R1.
• R2R-CE val-unseen results reported for the model: NE 3.86, OS 71.2, SR 69.8, SPL 66.9, compared with MobileVLA-R1 at NE 4.05, OS 69.7, SR 68.3, SPL 65.2.
• RxR-CE val-unseen results reported for the model: NE 3.71, SR 73.1, SPL 68.5, nDTW 77.6, compared with MobileVLA-R1 at NE 3.92, SR 71.5, SPL 66.8, nDTW 76.1.
• Comparison against strong baselines: Table II lists competing methods including CorrectNav (SR 65.1 on R2R-CE, SR 69.3 on RxR-CE), StreamVLN (SR 56.9 / 52.9), NaVILA (SR 54.0 / 49.3), ETPNav (SR 57.0 / 54.7), and HNR (SR 61.0 / 56.3); the paper states MobileVLA-R1 2.0 consistently outperforms strong VLA baselines across these settings.
• Real-world humanoid gain: a 10.0 point improvement in full-task success on real-world G1 mobile manipulation tasks over MobileVLA-R1, with robust long-horizon instruction following and closed-loop execution across platforms.
• Humanoid transfer without humanoid training data: the Unitree G1 is introduced only at evaluation time; manipulation primitives such as reaching, grasping, lifting, transporting, and placing are executed by fixed controller-side routines rather than predicted as joint-level trajectories by the VLA policy.
• Dataset scale: MobileVLA-CoT contains 18K episode-level, 78K step-level, and 38K navigation-level reasoning-annotated samples, totaling 134K instances, synthesized from 168K raw generations.
• Source data: R2R (50K samples), RxR (58K samples), and QUARD (262K samples); all reasoning and action supervision for training derives exclusively from these three datasets.
• Reasoning structure: each sample pairs a <think>...</think> reasoning trace with an <answer>...</answer> executable target, enabling the same framework to supervise both structured reasoning and action generation.
• Reward design in GRPO: a composite reward combines a movement reward (cosine similarity between normalized predicted and target velocity commands), a behavior reward (exact match on the discrete behavior primitive), and a format reward (conformance to the <think>/<answer> structure), weighted by λ_mov, λ_beh, and λ_fmt.
• Decoder frozen during RL: GRPO is offline, uses no environment interaction or online robot adaptation, and keeps the reasoning-conditioned action decoder fixed so it receives no gradient from the RL objective.
• Not reported in the provided content: exact QUARD benchmark scores, exact G1 success-rate figures, end-to-end latency numbers for the hybrid onboard–remote deployment, and the numeric results of the ablations (decoder architecture comparisons, reward-weight sensitivity, reasoning-granularity studies). The paper states these analyses are included in the journal version, but the specific values do not appear in the supplied text.

Methodology in Plain English

The team started by building a reasoning dataset. They took existing embodied datasets (R2R, RxR, QUARD) and used a model-agnostic "CoT data engine" instantiated with Gemini-2.5-Flash to generate structured reasoning traces at three temporal granularities: episode-level (what the whole trajectory was about), step-level (what to do right now given the current view and recent history), and navigation-level (how a global instruction breaks into sequential spatial decisions). A four-stage semi-automatic check — format validity, action consistency, safety, and semantic correctness — filtered 168K raw generations down to 134K validated samples, removing malformed tags, missing command fields, invalid action ranges, unsafe or irrelevant outputs, and hallucinated objects.

The model itself is a LLaVA-style multimodal architecture initialized from NaVILA. RGB, depth, and point-cloud observations are encoded separately, projected into a shared hidden space, given modality-type embeddings, and concatenated with language tokens before the backbone processes them. The backbone autoregressively produces the reasoning trace.

The distinctive piece is the reasoning-conditioned action decoder. Rather than reading numbers out of the generated text, it takes the hidden states of the observation tokens and the generated reasoning tokens, forms a joint context, and lets two learnable queries (one for locomotion, one for behavior) cross-attend to that context. One regression head outputs continuous planar velocities (Vx, Vy, ω); another outputs a softmax distribution over discrete task-level behavior primitives such as interaction, posture, or skill-switching commands. The robot's own low-level controller then turns those task-level targets into actual motor commands, so the VLA policy decides what to do while the controller decides how to do it on that specific body.

Training happens in two stages. First, supervised fine-tuning on the MobileVLA-CoT subsets: long-horizon reasoning alignment on episode- and navigation-level data, then reasoning-to-action alignment on step-level data, jointly optimizing the CoT loss plus L1 locomotion loss and cross-entropy behavior loss (masked by indicators when a sample lacks that form of supervision). Modality encoders stay frozen; projection layers, LoRA parameters, and the decoder are trained. Second, offline GRPO: for each observation-instruction pair the policy samples multiple candidate outputs, each candidate's reasoning representation is passed through the fixed decoder to produce an action, and that action is scored with movement, behavior, and format rewards. Group-relative advantages plus KL regularization to a frozen reference policy then update the VLA policy.

Why This Matters

• Impact on research: The paper argues that prior embodied-reasoning work mostly produces human-interpretable rationales or intermediate representations without an explicit bridge to executable System-1/System-0 control. It supplies that bridge in the form of a learnable decoder and shows that task-level action abstraction is a viable boundary between semantic decision making and embodiment-specific actuation — a design that sidesteps learning morphology-specific joint dynamics.
• Real-world applications:
  • Indoor service and delivery robots that follow natural-language instructions through buildings (language-guided navigation, VLN-CE settings).
  • Quadruped robots performing language-conditioned locomotion and interaction tasks (QUARD-style embodied continuous control).
  • Humanoid mobile manipulation in tabletop, shelf/cabinet, and cluttered environments, including object search, approach, grasping, lifting, transporting, and placing.
  • Long-horizon composite tasks where navigation and physical interaction must be chained within a single instruction.
• Industry relevance: The embodiment-decoupled interface means a company could train one policy and deploy it across a fleet of different robot bodies by swapping low-level controllers, without collecting platform-specific demonstrations. The explicit trade-off is that manipulation execution quality is delegated to fixed controller-side routines, so the policy governs task-level intent rather than joint-level dexterity. The use of offline GRPO also matters practically, since it avoids risky online exploration on physical hardware.

Future Directions

• Online and interactive RL: The current GRPO stage is offline on a fixed dataset with no environment interaction or online robot adaptation. Extending to interactive optimization on real hardware is a natural next step.
• Learning the manipulation routines rather than fixing them: Manipulation primitives on the G1 are executed by fixed controller-side routines. Replacing these with learned, still transferable policies would test how far the embodiment-decoupling idea can be pushed.
• Richer behavior primitive vocabularies and evaluation: The framework's expressiveness is bounded by the predefined discrete behavior set A_beh; scaling and evaluating larger primitive sets across platforms remains open.
• Understanding the reported failure modes: The paper states that G1 failures are decomposed into grounding, navigation, grasping, manipulation-execution, and low-level control errors, and that episode-level failure modes on Go2 are analyzed. Turning those diagnostics into targeted improvements — particularly distinguishing reasoning failures from controller failures — is an open question.
• Deeper reasoning-supervision studies: The journal version reports analyses of reasoning-supervision granularity, multimodal perception, rationale sources, policy-optimization objectives, and reward-weight sensitivity; how these factors interact across embodiments is not yet settled.

Target Audience

Robotics and embodied-AI researchers working on vision-language-action models, language-guided navigation, and mobile manipulation; reinforcement-learning researchers interested in applying GRPO-style optimization to robot policies; and engineers building generalist robot stacks who need a policy that transfers across quadruped and humanoid platforms without per-embodiment demonstration data. Readers should be comfortable with multimodal transformer architectures, Chain-of-Thought supervision, and policy-gradient methods.

Authors’ abstract

Grounding natural-language instructions into reliable and executable actions remains a fundamental challenge for vision-language-action (VLA) systems on mobile robots, due to the persistent gap between high-level semantic reasoning and low-level locomotion and manipulation control. Existing approaches often rely on implicit reasoning or monolithic action prediction, making it difficult to maintain coherent long-horizon decision making while producing precise and adaptable robot actions. To address this challenge, we propose MobileVLA-R1 2.0, an RL-enhanced VLA framework that explicitly couples structured embodied reasoning with executable mobile robot control. The framework learns multi-granularity reasoning over embodied trajectories through supervised Chain-of-Thought (CoT) alignment and reinforcement learning, improving reasoning-to-action consistency beyond purely behavioral supervision. To support both locomotion and manipulation, we further introduce a reasoning-conditioned action decoder that maps multimodal reasoning representations to task-level action targets, which are subsequently translated into embodiment-specific commands by robot controllers. This design provides a unified perception-reasoning-action interface while decoupling high-level action generation from robot-specific actuation. We conduct extensive evaluations on language-guided navigation, quadruped control, and humanoid mobile manipulation, covering VLN-CE, QUARD, and real-world deployments on Unitree Go2 and G1 robots. MobileVLA-R1 2.0 consistently outperforms strong VLA baselines, achieving an average 1.6 point improvement in SR on VLN-CE and a 10.0 point improvement in full-task success on real-world G1 mobile manipulation tasks over MobileVLA-R1, while demonstrating robust long-horizon instruction following and closed-loop execution across different robotic platforms.