Embodied Navigation with Auxiliary Task of Action Description Prediction

AI summary

Overview

Research area: Embodied AI / multimodal robot navigation — combining reinforcement learning (RL) and imitation learning (IL) for object-goal navigation (ObjNav), vision-and-language navigation (VLN), and semantic audio-visual navigation (SAVNav), with natural-language action description generation as an auxiliary task.

Technical level: Advanced. The paper assumes familiarity with reinforcement learning, transformer architectures, knowledge distillation, and navigation benchmarks such as Habitat, SoundSpaces, and Matterport3D.

Scope: The paper proposes "descriptive reinforcement learning" (DescRL), an auxiliary action-description-prediction task added to existing navigation agents, and evaluates it across three navigation tasks and several baselines.

What This Paper Is About

Multimodal navigation agents are becoming more capable but also more opaque, and existing explainable approaches generally accept a drop in task performance as the price of explanation. The problem is that in reinforcement learning there is no ground-truth sentence describing what an agent is doing, so describing actions cannot simply be trained as supervised data the way it can in imitation learning. The authors solve this by distilling descriptions from pre-trained description generators (including vision-language models) into the RL agent as an auxiliary task, so the agent both explains itself and navigates well.

Key Contributions

1. A combined descriptive and performant RL method. The paper proposes DescRL, which trains navigation and action description prediction jointly, arguing that the two tasks are compatible rather than in conflict.
2. Removal of the human-annotation dependency. By distilling from pre-trained models — a CNN+transformer ADGenerator trained on R2R, or off-the-shelf VLMs such as VideoLLaMA2 and Qwen2.5-VL — the method can generate description targets without human-created sentences for each new task.
3. A comprehensive experiment over description types and auxiliary tasks. The authors compare past action description (P-AD), future action description (F-AD), and past-future action description (PF-AD), and benchmark against alternative auxiliary tasks: next action, progress, next frame, next spectrogram, goal location, and goal category.
4. Cross-task validation with a state-of-the-art result. The method is applied to ObjNav, VLN, and SAVNav, and reports state-of-the-art performance on all SAVNav evaluation metrics used.

Main Findings

• SAVNav state-of-the-art: Applied to SAVi, P-DescRL reached SR 37.4, SPL 32.4, SNA 28.0, DTG 8.4, SWS 19.1 in the Heard setting and SR 31.4, SPL 26.9, SNA 22.5, DTG 8.7, SWS 15.1 in the Unheard setting (Table 3), above the reported SAVi baseline (SR 31.6 / 24.7) and KSAVEN (SR 25.1 / 21.1).
• Past descriptions beat future descriptions: Learning to predict P-AD had a better navigation impact than F-AD. F-DescRL on SAVi scored SR 36.4 (Heard) but dropped to SR 22.5 (Unheard), below the SAVi baseline of 24.7; the authors attribute this to future prediction being intrinsically hard when goal categories are hard to infer.
• Object-goal navigation gains: SMT improved from SR 17.9, SPL 7.7, DTG 6.72 to SR 26.7, SPL 9.7, DTG 5.91 with pre-trained Past-DescRL, versus GRU-based at 7.4 / 4.2 / 6.49. Without ADPredictor pre-training, Past-DescRL scored only 12.8 / 5.7 / 6.95.
• VLN gains are real but small: On DUET, Past-DescRL changed Val Unseen from NE 3.21 / SR 71.65 / SPL 60.44 to NE 3.09 / SR 72.33 / SPL 61.37 and Test Unseen from 3.63 / 69.76 / 59.39 to 3.56 / 69.88 / 59.40. On ScaleVLN the change was still smaller; the authors state that an auxiliary method's contribution may be small when the task is simple or the baseline already performs well.
• Action description is a better auxiliary task than the alternatives tested: In Table 4, P-DescRL outperformed next action, progress, next frame, next spectrogram, goal location, and goal category on most metrics.
• Pre-training the ADPredictor matters: Ablations (Tables 1 and 6) consistently showed better performance with the phase-2 pre-training step; the authors attribute this to learning grammar and other non-navigation skills in advance.
• Task embeddings help: Comparing rows 5 and 6 of Table 6, task embeddings improved most metrics, which the authors say matters for distinguishing when the agent is choosing an action versus predicting a description.
• Sharing decoders is a trade-off: With three decoders, sharing two layers performed best on many SAVNav metrics (SR 37.4, SPL 32.4, SNA 28.0 Heard). Sharing zero layers reduced the benefit to navigation; sharing all three forced the decision latent space and description latent space to coincide. In ObjNav, performance was higher with fewer shared decoders.
• VLMs can substitute for human data, with caveats: Using VideoLLaMA2 with no fine-tuning gave SR 33.7 / SPL 29.8 (Heard) versus 31.6 / 28.5 for no ADGenerator at all, showing gains without human-created data. Fine-tuning VideoLLaMA2 on R2R lowered results to 28.9 / 25.6, which the authors attribute to overfitting, and using the stronger Qwen2.5-VL did not necessarily improve performance (33.4 / 28.6).
• Failure analysis: The authors report a domain gap between VLN and SAVNav — success in VLN is judged at 3 m while SAVNav uses 1 m — leading to failures where the agent approached but did not stop at the exact position. In a VLN case, the agent stopped at the 7th step instead of the 5th and the descriptions showed it never attended to the floor circle.
• Reported limitations: Biased common-sense in the dataset (e.g., "stairs" appearing in hallways with no stairs), stopping at the correct category but wrong instance, untested transfer to other RL tasks or Embodied Question Answering, and the assumption that the robot matches the roughly human-sized robot assumed in R2R data.

Methodology in Plain English

The method runs in two phases.

Phase 1 — train a describer. An "ADGenerator" is trained to turn what the agent sees into a sentence. For ObjNav and SAVNav it takes a sequence of actions as one-hot vectors in {0,1}^4 plus visual observations of size 128x128x7 (RGBD plus semantic images), passes the visuals through a CNN producing 512-dimensional features, concatenates actions and features, and outputs words through a transformer encoder-decoder. For VLN it operates over graph nodes using an encoder like DUET's coarse-scale cross-modal encoder. It was trained on R2R with teacher forcing and cross-entropy loss, using 10,819 training and 1,839 evaluation examples, with GloVe embeddings for ObjNav/SAVNav and a BERT tokenizer for VLN. Alternatively, this describer can be a VLM (VideoLLaMA2 or Qwen2.5-VL) receiving RGB images of size 336x336x3 plus a prompt, or that same VLM fine-tuned with QLoRA for 1 epoch at learning rate 2.0e-5, LoRA r=128, LoRA alpha=256.

Phase 2 — train navigation with description prediction as an auxiliary task. An "ADPredictor" is attached to the navigation agent — for SAVNav it is a transformer decoder fed the agent's predicted goal location and category as the start of the sentence. The transformer encoder and decoder are shared between the policy and the predictor, with task embeddings distinguishing the two roles. The agent minimizes L_RL + lambda * L_CE with lambda = 0.1. Training has two steps: first the predictor alone is trained to learn the observation encoder and shared encoder/decoder; then navigation and description prediction are learned together, with the predictor always in teacher-forcing mode. Because R2R is unavailable for ObjNav and SAVNav, new data was generated by taking shortest paths between random starts and goals and running the phase-1 ADGenerator over them, producing roughly 100k examples for ObjNav and 500k for SAVNav. For future descriptions, observations V_t through V_{t+k} are fed to the generator at train time only; at test time the predictor sees only past observations, which the authors say keeps comparisons fair.

Variant naming. P-DescRL, F-DescRL, and PF-DescRL indicate whether the generator receives past, future, or both sets of observations.

Training setups. ObjNav used 8 GPUs with 10 processes each (80 processes), DD-PPO, 3,000 parameter updates, 2 transformer encoder layers, 2 shared decoder layers, 1 unshared decoder layer in both policy and predictor, and generator input length k+1 = 20. VLN used 1 GPU and 1 process, DAgger imitation learning, 0 shared decoder layers, 3 unshared predictor decoder layers, and the full visual observation history. SAVNav used 4 GPUs with 8 processes each (described as 36 processes total), with other DescRL settings matching ObjNav.

Evaluation setups. ObjNav used the Habitat Simulator with Matterport3D, floors averaging 512 m^2, 67 scenes split 56/4/7, reward r_t = alpha * I_goal + (d_t - d_{t-1}) + r_penalty with alpha = 2.5 and penalty -0.001, measured by SR, SPL, and DTG. VLN used Matterport3DSimulator with the same setup as prior work, measured by NE, SR, SPL. SAVNav used the SoundSpaces simulator with Matterport3D, 102 scenes split 73/11/18, reward r_t = alpha * I_goal + I[d_t > d_{t-1}] + r_penalty with alpha = 10 and penalty -0.01, measured by SR, SPL, SNA, DTG, SWS, averaged over 1,000 tests. Description decoding used greedy search in VLN and top-k/top-p sampling with temperature 2.0, k = 10, p = 0.95 in SAVNav.

Why This Matters

Impact on research. The paper challenges the assumption that explainability must cost navigation performance, and shows that a description task can improve the main objective rather than degrade it. It also offers a route around the missing-ground-truth problem in RL by distilling targets from pre-trained models, which the authors note is lightweight compared to foundation models and capable of running in real time, while avoiding the catastrophic forgetting that they say afflicts fine-tuned foundation-model navigation methods. The paper also reports that the KSAVEN baseline was re-run using only heard sounds rather than all sounds, for realism and fairness.

Potential real-world applications (plausible implications; the paper reports no real-robot deployment):

• Home or warehouse robots that announce what they just did or intend to do next, giving operators a running narrative for oversight.
• Failure diagnosis: the qualitative analysis shows descriptions exposing that an agent stopped early or ignored a visual cue, which supports debugging rather than only after-the-fact scoring.
• Search-and-rescue or assistive robots navigating toward sounds, where the task studied (SAVNav) requires reaching a sounding target even after the sound stops.
• Instruction-following service robots, where the VLN experiments show a description task can be added without harming instruction-following accuracy.

Industry relevance. The method is applied on top of existing agents (SMT, DUET, Scale

Authors’ abstract

The field of multimodal robot navigation in indoor environments has garnered significant attention in recent years. However, as tasks and methods become more advanced, the action decision systems tend to become more complex and operate as black-boxes. For a reliable system, the ability to explain or describe its decisions is crucial; however, there tends to be a trade-off in that explainable systems can not outperform non-explainable systems in terms of performance. In this paper, we propose incorporating the task of describing actions in language into the reinforcement learning of navigation as an auxiliary task. Existing studies have found it difficult to incorporate describing actions into reinforcement learning due to the absence of ground-truth data. We address this issue by leveraging knowledge distillation from pre-trained description generation models, such as vision-language models. We comprehensively evaluate our approach across various navigation tasks, demonstrating that it can describe actions while attaining high navigation performance. Furthermore, it achieves state-of-the-art performance in the particularly challenging multimodal navigation task of semantic audio-visual navigation.