CompassNav: Steering From Path Imitation To Decision Understanding In Navigation

AI summary

Overview

• Research area: Embodied AI, specifically goal-driven navigation with Large Vision-Language Models (LVLMs), including reward design for reinforcement fine-tuning.
• Technical level: Advanced. The paper assumes familiarity with reinforcement learning from human feedback-style pipelines (SFT, GRPO, KL regularization), vision-language model training, and embodied navigation benchmarks.
• Scope: The paper introduces a dataset (Compass-Data-22k) and a Gap-Aware Hybrid Reward function to train a 7B LVLM navigator that evaluates all feasible actions rather than imitating a single expert path, evaluated on ObjectNav, Instance-Image-Goal Navigation, NavNuances, HM3D-OVON, and AI2-THOR (ProcTHOR), plus real-world robot deployment.

What This Paper Is About

Most LVLM navigation agents are trained by imitating one expert trajectory, which turns navigation into memorizing a single correct path and ignores the many other valid routes that exist. The authors argue this is the wrong objective and propose instead teaching agents "Decision Understanding": at each step, the agent should assess the relative quality of every feasible move and reason about why one action beats another. To do this, they build a dataset where all candidate actions are annotated with A* geodesic distances to the goal, and a reward function that adapts its feedback depending on how clear-cut the decision is.

Key Contributions

1. Compass-Data-22k dataset. A 22k-trajectory dataset for decision understanding, containing distilled reasoning traces for Supervised Fine-Tuning (a Compass-Data-SFT-11k set) and an RFT subset that densely annotates all feasible actions with A* geodesic distances, effectively mapping the gradient of correctness across the decision space.
2. A Gap-Aware Hybrid Reward function. A reward that dynamically adapts its feedback to decision certainty, providing decisive signals for optimal actions in unambiguous states and nuanced, non-extreme scores in ambiguous ones.
3. The SFT-then-RFT training recipe. A two-stage pipeline that first overcomes the "cold-start" problem via imitation of a teacher's reason-then-act traces, then aligns the policy with the Gap-Aware Hybrid Reward using GRPO.
4. A 7B state-of-the-art navigator. CompassNav, built on Qwen2.5-VL-7B, which the authors report sets a new state-of-the-art on goal navigation benchmarks, outperforming larger proprietary models, and achieves real-world goal navigation on a physical robot.

Main Findings

• Outperforms modular systems on HM3D and MP3D. CompassNav reaches SR 56.6 / SPL 27.6 on HM3D and SR 42.0 / SPL 17.5 on MP3D, versus UniGoal at 54.5 / 25.1 and 41.0 / 16.4, SG-Nav at 54.0 / 24.9 and 40.2 / 16.0, and VLFM at 52.4 / 30.4 and 36.4 / 17.5. Notably, CompassNav is the only method in that table marked as both End-to-End and without memory.
• Beats larger proprietary LVLMs on navigation. On ObjNav / InsImageNav / AVG, CompassNav (SFT+RFT) scores 61.6 / 27.8, 35.6 / 14.8, and 48.6 / 21.3, compared with GPT-4o at 52.4 / 23.5, 29.8 / 13.2, 41.1 / 18.4; Gemini-2.5-Flash at 50.0 / 10.6, 24.1 / 9.70, 37.1 / 10.2; and GPT-o4-mini at 59.6 / 26.9, 33.4 / 13.2, 46.5 / 20.1.
• The RFT stage is what drives the large gain. The base model scores 19.8 / 5.20 on InsImageNav; SFT raises it to 23.3 / 7.90, and adding RFT raises it to 35.6 / 14.8.
• RFT only works after SFT initialization. RFT trained from scratch yields a marginal 3.7 improvement in SR (19.8 to 23.5), while RFT on the SFT-initialized model improves by another 12.3 (23.3 to 35.6). SFT with action-only supervision actually degrades performance to 17.9 / 5.78.
• Large gains on atomic spatial reasoning (NavNuances). CompassNav-7B records DC 82.54, NU 23.85, LR 59.92, RR 59.31, VM 22.94, versus the base Qwen2.5-VL-7B at 79.62, 19.10, 50.37, 49.00, 7.88. The VM (Vertical Movement) score is described as roughly a 3x improvement over the base model, and it exceeds NavGPT-4V's 13.64.
• Remaining gap versus GPT-4V on instruction-following metrics. CompassNav stays below NavGPT-4V on DC (82.54 vs 92.68) and NU (23.85 vs 39.13), which the authors attribute to task alignment (goal-oriented exploration versus rigid VLN instruction following) and to 7B-scale models being more prone to hallucination in memory-dependent long-context tasks.
• Reward function comparisons. Within the ablation setting, the Gap-Aware Hybrid Reward achieves 35.6 / 14.8, against Binary at 29.5 / 11.1 and Min-Max at 29.2 / 12.5.
• Reward behavior across scenarios. In the Decisive Case (A), the method produces a margin of 1.00 versus 0.12 between best and second-best actions; in the Indistinguishable Case (C), it returns an honest low score of 0.25 where other methods return a misleading 1.0.
• Hyperparameter sensitivity. Max bonus B=1.0 (35.6 / 14.8) beats B=0.5 (31.3 / 12.2) and B=1.5 (27.9 / 11.7); temperature T=0.5 (35.6 / 14.8) beats T=0.2 (33.2 / 13.3) and T=0.8 (33.7 / 13.9). GRPO group size K=5 (61.6 / 27.8) is preferred over K=3 (60.2 / 26.1) and K=7 (61.8 / 26.0). A fixed KL coefficient (61.6 / 27.8) slightly edges an adaptive KL mechanism (61.5 / 26.0) with kl_target=0.5 and kl_horizon=2560.0.
• Efficiency versus Nav-R1. On HM3D-OVON val-unseen, CompassNav scores 43.5 / 21.6 versus Nav-R1 at 42.2 / 20.1, Uni-NaVid at 39.5 / 19.8, and MTU3D at 40.8 / 12.1, while using one-tenth of the training data and starting from a general-purpose LVLM rather than a 3D-specialized model.
• Sim-to-sim generalization. On AI2-THOR (ProcTHOR), with over 8,000 episodes in unseen synthetic environments, CompassNav scores 58.1 / 32.8 versus ProcTHOR at 55.0 / 23.7 and EmbCLIP at 47.0 / 20.0; the SPL gain of +9.1 over the ProcTHOR agent is highlighted as evidence of transferable spatial logic.
• Real-world deployment is claimed but not quantified here. The abstract and conclusion state robust real-world goal navigation on a physical robot; the provided content does not report numerical real-world results.

Methodology in Plain English

The authors reframe navigation as a sequential decision problem: at each step the agent sees its current view plus a set of feasible candidate actions and must pick one. A component called the Action Proposal Module (APM) uses real-time depth maps and occupancy grids to propose safe, visually distinct candidate actions as polar vectors (r, θ), filtering out regions already marked explored within a 2-meter radius. An Oracle A* Annotator then uses global simulator information to compute the shortest geodesic distance to the goal for every candidate, producing a full action-value vector per timestep instead of a single "correct" action. A Backtracking Mechanism deliberately revisits ambiguous points (states with multiple viable options) to record alternative trajectories.

Training proceeds in two stages. Stage 1 is Supervised Fine-Tuning on the 11k SFT set, whose targets are the full reasoning and action traces of a teacher model, Qwen-QvQ, recorded only from its successful episodes in habitat-sim; outputs follow a <think>...</think><answer>k</answer> format. Masked multiple-choice decoding applies a masked softmax over the decoder logits so the chosen action index is always one of the valid candidates. Historical frames are deliberately excluded from the SFT input to keep the design compatible with external memory modules, because the authors found that feeding historical context (text summaries or concatenated images) consistently degraded performance.

Stage 2 uses GRPO. For each prompt, the policy samples a group of G sequences; each sequence's chosen action is scored by the Gap-Aware Hybrid Reward. That reward combines a continuous base score — a softmax over goal distances with temperature τ — with a dynamic bonus triggered only for the optimal action, scaled by a certainty factor g_t. Certainty is computed as the clipped, normalized gap between the best and second-best distances, so the signal encourages broad exploration far from the goal and precision near it. A KL term against the frozen SFT policy regularizes updates. The whole agent is built on Qwen2.5-VL-7B, and a separate, non-fine-tuned Qwen2.5-VL-7B acts as a dedicated Stop Agent prompted at each step.

Training data is generated in habitat-sim on the HM3Dv2 train split, and evaluation uses the completely held-out HM3Dv1-val, HM3Dv2-val, and MP3D-val splits, with primary tasks being ObjectNav and Instance-Image-Goal Navigation. Metrics are Success Rate (SR) and Success weighted by Path Length (SPL). An episode counts as successful if the agent issues stop within a 1-meter radius of the goal in under 500 steps; the low-level controller converts arrows into move_forward (25 cm), turn_left (30°), and turn_right (30°).

Why This Matters

The paper argues that the field's default training signal — imitate one expert path — teaches memorization rather than reasoning, and shows that switching the supervision to a panoramic, per-action value assessment plus a certainty-adaptive reward produces a 7B model that outperforms much larger proprietary systems. It also demonstrates that a general-purpose open-source LVLM with one-tenth the training data of a concurrent 3D-specialized effort can beat it, which matters for the cost and accessibility of embodied agents. The authors commit to releasing the dataset, LoRA weights, and all code for data generation, training, and physical deployment.

Real-world applications implied by the work:

• Assistive and service robotics. The ethics statement explicitly frames the goal as supporting transparent research in assistive robotics, where a user gives a sparse goal like "find a chair."
• Semantic, open-ended commands. The paper contrasts engineered maps with the difficulty of interpreting unstructured human requests such as "find a quiet spot for reading," which LVLMs handle natively.
• Multi-floor and complex indoor environments. The Vertical Movement (VM) results target commands like "Go upstairs," relevant to home and office robots spanning floors.
• Search and retrieval tasks in unfamiliar buildings. Object-goal and instance-image-goal navigation transfer to finding a specific object instance from a reference photo.

Industry relevance: the paper positions end-to-end LVLMs as avoiding the fragility to sensor noise, pose drift, and error propagation of modular SLAM-based pipelines, and as avoiding the prohibitive cost and latency of closed-source models like GPT-4o for real-time control. It presents a cost-effective open-source alternative (7B parameters, LoRA weights released) that lowers the barrier to deploying advanced embodied agents, and the AI2-THOR results suggest transfer to non-photorealistic, game-like simulated domains.

Future Directions

1. Closing the instruction-following gap. CompassNav remains behind GPT-4V on Direction Change and Numerical Comprehension, which the authors attribute to task alignment and model scale; narrowing this gap, possibly by reconciling goal-oriented exploration with rigid VLN instruction following, is an open problem.
2. Mitigating 7B-scale hallucinations in memory-dependent tasks. The lower Numerical Comprehension score is attributed to hallucination in long-context, memory-dependent settings, implying a need for better memory mechanisms or larger foundations.
3. Integrating external memory. The authors deliberately exclude historical frames from the SFT input for compatibility with external memory modules, but report that current methods for baking historical context into training (text summaries, concatenated images) degrade performance — leaving better memory integration unresolved.
4. Scaling and further validating the real-world claim. The paper states robust real-world robot navigation but the provided content contains no quantitative deployment results; broader real-world evaluation, and extending the dense A* annotation pipeline to more domains, are natural next steps.

Target Audience

Researchers and engineers working on embodied AI, vision-language-action models, and reinforcement fine-tuning for long-horizon decision problems; practitioners building robot navigation stacks who want an end-to-end alternative to modular SLAM pipelines; and reinforcement learning researchers interested in reward shaping under uncertainty and ambiguity, since the Gap-Aware Hybrid Reward design transfers conceptually beyond navigation.

Authors’ abstract

The dominant paradigm for training Large Vision-Language Models (LVLMs) in navigation relies on imitating expert trajectories. This approach reduces the complex navigation task to a sequence-to-sequence replication of a single correct path, fundamentally limiting the agent's ability to explore and generalize. In this work, we argue for and introduce a new paradigm: a shift from Path Imitation to Decision Understanding. The goal of this paradigm is to build agents that do not just follow, but truly understand how to navigate. We materialize this through two core contributions: first, we introduce Compass-Data-22k, a novel 22k-trajectory dataset. Its Reinforcement Fine-Tuning (RFT) subset provides a panoramic view of the decision landscape by annotating all feasible actions with A* geodesic distances. Second, we design a novel gap-aware hybrid reward function that dynamically adapts its feedback to decision certainty, shifting between decisive signals for optimal actions and nuanced scores to encourage exploration. Integrated into an SFT-then-RFT recipe, our CompassNav agent is trained not to memorize static routes, but to develop an internal compass that constantly intuits the direction to the goal by evaluating the relative quality of all possible moves. This approach enables our 7B agent to set a new state-of-the-art on Goal navigation benchmarks, outperforming even larger proprietary models, and achieve robust real-world goal navigation on a physical robot.