Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models

AI summary

Overview

• Research area: Computer vision and multimodal learning — specifically spatial reasoning in vision-language models (VLMs), combining reinforcement learning (GRPO) with on-policy self-distillation (OPSD).
• Technical level: Advanced. The paper assumes familiarity with policy-gradient RL, token-level KL divergence, privileged-information distillation, and 3D scene representations (depth, semantic segmentation, bird's-eye-view layouts).
• Scope: The paper proposes GPD (Geometry-Privileged Distillation), a training framework that feeds question-routed 3D geometric cues to a teacher model during training so that an RGB-only student model learns better spatial reasoning, and evaluates it on five spatial reasoning benchmarks with Qwen3-VL-Instruct backbones at 2B and 4B scales.

What This Paper Is About

Vision-language models are weak at judging distance, direction, and layout because RGB images do not directly contain geometric evidence — the model has to infer depth and 3D structure from appearance alone. Prior fixes either bolt 3D encoders or perception tools onto the model at inference time (adding architecture and latency costs) or train with outcome rewards that only score the final answer, leaving a misjudged depth or a flipped left-right relation uncorrected. GPD's goal is to use the geometric information that is already available in 3D-scanned training corpora as a training-time privilege for a teacher model, while keeping the deployed student model RGB-only.

Key Contributions

1. A formulation of geometry-aware privileged self-distillation for spatial reasoning. The teacher is equipped with spatially grounded evidence rather than only the reference answer, which enables meaningful token-level distillation along the student's incorrect reasoning trajectories.
2. The GPD method itself. It equips an OPSD teacher with question-routed depth, semantic, and BEV textual cues, concentrates token-level distillation on incorrect trajectories via error gating, and retains RGB-only inference at deployment.
3. An empirical demonstration across two model scales and five benchmarks. GPD gives consistent improvements over GRPO and answer-privileged OPSD (Qwen3-VL 2B and 4B backbones), with ablations validating the design choices (cue complementarity, routing vs. full injection, incorrect-only distillation, and the KL weight).
4. Evidence of privilege internalization. Under RGB-only inference, GPD (56.3) already beats GRPO (54.9), and gains less when routed 3D text is injected at test time (+0.6 vs. +2.1 for GRPO).

Main Findings

• VSI-Bench, 4B backbone: GPD reaches the highest controlled average of 57.1, improving over the base model by 1.7 points, over GRPO by 0.7 points, and over OPSD by 1.4 points (base 55.4, GRPO 56.4, OPSD 55.7).
• VSI-Bench, 2B backbone: GPD reaches the best controlled average of 51.7, beating the base model and GRPO by 0.4 points and OPSD by 0.9 points (base 51.3, GRPO 51.3, OPSD 50.8).
• Four additional benchmarks, 4B: GPD achieves the highest controlled average of 37.6 across MindCube, SPAR-Bench, MMSI-Bench, and ViewSpatial — 1.9 points over GRPO and 6.1 points over OPSD. Per-benchmark: MindCube 38.0 vs. 33.0 for GRPO (+5.0), SPAR 43.4 vs. 41.4 (+2.0), MMSI 29.4 vs. 28.7 (+0.7), ViewSpatial 39.8.
• Four additional benchmarks, 2B: GPD achieves the best controlled average of 33.6, ahead of GRPO by 0.9 points and OPSD by 3.4 points, with the largest gain on MindCube (38.4 vs. 35.8 for GRPO, +2.6 points).
• Answer and 3D privilege are complementary: on Qwen3-VL-4B, answer alone reaches 40.2 (+0.4 over GRPO), routed 3D text alone reaches 40.5 (+0.7), and combining both reaches 41.5.
• Routing beats indiscriminate injection: full 3D text injection scores 39.8 — 1.7 points below routed text (41.5) despite containing more information. Routed images score 40.4, 1.1 points below routed text.
• Distillation only on incorrect trajectories is better: applying privileged KL to all trajectories lowers the average by 0.9 points to 40.6, consistent with noisy teacher signals disturbing already-correct trajectories.
• KL weight is sensitive: among incorrect-only variants, the average rises from 39.6 at λ=0.001 to 41.5 at λ=0.003 (+1.7 over GRPO), then falls to 40.2 at λ=0.004 and 39.6 at λ=0.005.
• Zero-shot privilege form study (1000-question VSI-590K subset, Qwen3-VL-4B-Instruct): the RGB-only baseline is 49.1; full 3D images drop to 47.4; routed images and full text both reach 48.3; only routed text exceeds the baseline, at 52.2 (+3.1 points).
• Routing distribution matches geometric intuition: object counting is routed entirely to semantic maps, absolute distance relies predominantly on depth, room size is dominated by BEV, and relative direction uses both semantic and BEV.
• Training dynamics: GPD and GRPO show similar validation accuracy early, but GPD pulls ahead while GRPO plateaus; GPD reaches a higher entropy peak earlier, indicating broader early exploration.
• Comparison against published systems on VSI-Bench: GPD's 57.1 exceeds general-purpose VLMs such as GPT-4o (34.0), LLaVA-OneVision-72B (40.3), Qwen2.5-VL-7B (35.8), and Gemini-3-Pro (52.5), as well as 3D-augmented Spatial-MLLM-4B (47.3) and Think3D-4B (average not reported; only per-subtask scores given), and outcome-supervised SpaceR-7B (44.5) and SpatialLadder-3B (45.7).

Methodology in Plain English

The method has two stages.

Stage 1 — build the privilege offline. For every training example (RGB observations, question, reference answer), the authors assemble three kinds of 3D cue and turn them into short text summaries: depth (object distances and near-to-far order), semantics (a list of objects with their image regions), and BEV (top-down spatial relations). Where aligned 3D annotations exist — the VSI and SPAR data are derived from ScanNet scenes — the cues come directly from those annotations. Where they do not (MindCube), depth is estimated with Depth Anything 3 and semantic instances with Grounded-SAM-2; BEV is included only when reliable camera poses and layout are available. The cues are computed once and fixed for the whole run. Then a base VLM routes only the question-relevant subset of cues (e.g., counting questions to semantics, distance questions to depth and BEV) to build a scene context, which is concatenated with the reference answer to form the teacher's privilege.

Stage 2 — train student and teacher as the same model. The student sees only RGB plus the question and samples a group of G responses. A group-normalized advantage from the outcome reward drives the standard GRPO objective. The same VLM, additionally conditioned on the routed 3D cues and reference answer, re-scores those sampled tokens. The teacher-student log-probability gap at each token gives a cheap single-sample reverse-KL estimate, avoiding a full-vocabulary summation. Two refinements matter: the distillation loss is gated to incorrect trajectories only (reward below τ = 0.5), so correct answers are left alone; and the aggregate loss over a micro-batch is clipped at 1.5 before being scaled by λ and added to the GRPO loss. The deployed model never sees the 3D cues. Training uses 10K VSI, 4K SPAR, and 1K MindCube samples, one epoch, a learning rate of 1×10⁻⁶, rollout group size n = 8, and λ = 0.003.

Why This Matters

Impact on research. The paper reframes a trade-off that has structured the spatial-reasoning literature: explicit 3D methods buy geometric evidence with inference cost, while training-based methods keep inference cheap but supervise only outcomes. GPD shows that geometry can be used purely as a training signal, and that what the teacher is privileged with matters — the answer alone corrects little, because spatial errors originate in perception. The zero-shot finding that routed 3D text (52.2) beats raw 3D images (47.4) and full text (48.3) against a 49.1 baseline is a concrete design lesson for anyone building privileged teachers.

Real-world applications:

• Robotics and manipulation: grasping the nearest object or judging reachability, where a wrong depth order or flipped left-right relation causes failure.
• Indoor navigation and embodied agents: route planning and approach-order decisions in unfamiliar rooms, precisely the subtasks where routed BEV and depth cues were selected.
• Assistive and mixed-reality guidance: describing how far a sofa is from a door, or which direction to turn, from ordinary camera footage without depth sensors.
• Multimodal assistants on consumer devices: phone- or edge-deployed VLMs where adding a 3D encoder or test-time reconstruction pipeline is not affordable, but the RGB-only interface is preserved.

Industry relevance. Because GPD requires no architectural change and no extra inference cost, it fits existing VLM training pipelines. The training data are 3D-scanned corpora that teams may already have, or off-the-shelf estimators (Depth Anything 3, Grounded-SAM-2) where annotations are missing. The cost of 3D supervision is paid once, offline, rather than at every query.

Future Directions

• Extending routing and privilege forms. The paper tests text, images, and full-context injection; whether richer privilege (e.g., point clouds or rendered novel views) can beat routed text is not reported.
• Generalizing beyond the evaluated corpora and backbones. GPD is evaluated only with Qwen3-VL-Instruct at 2B and 4B on five benchmarks; behavior at larger scales or with other VLM families is not reported.
• Automating or improving the router. Routing currently uses the base VLM and is validated by showing the distribution matches geometric intuition; whether a learned or self-improving router does better is left open.
• Understanding the internalization mechanism. GPD shows a smaller privilege gap (56.3 RGB-only, +0.6 with test-time 3D text, versus GRPO's 54.9 and +2.1), which the authors interpret as internalization; a mechanistic account of how geometric cues are absorbed into the policy is not provided.
• Sensitivity and stability of the distillation weight. The λ sweep shows a narrow optimum around 0.003, with performance dropping at 0.004 and 0.005; more robust scheduling or adaptive weighting is an open question.

Target Audience

• Multimodal and embodied-AI researchers working on spatial reasoning, 3D-grounded VLMs, or vision-language navigation.
• RL and post-training engineers interested in GRPO, on-policy distillation, and privileged-information training for large models.
• Practitioners deploying VLMs on hardware where adding 3D sensors, encoders, or test-time reconstruction pipelines is impractical.
• Graduate students and advanced readers with background in reinforcement learning and multimodal models; the token-level KL derivations and the staged pipeline make the paper hard going for beginners without that foundation.

Authors’ abstract

Spatial reasoning remains a persistent weakness of vision-language models (VLMs), because RGB inputs do not directly provide geometric evidence. Existing remedies either inject 3D into the model at inference, paying architecture and latency costs, or train with outcome rewards that supervise only the final answer. Spatial errors originate in perception: a misjudged depth or direction can be corrected only by the scene's true geometry, which the 3D-scanned sources of spatial training corpora already provide. We propose GPD (Geometry-Privileged Distillation), which makes geometric evidence the privilege in on-policy self-distillation (OPSD). For each question, depth, semantic, and bird's-eye-view (BEV) cues are rendered as compact text and routed to the teacher alongside the reference answer; a privileged KL, applied only to incorrect trajectories, augments GRPO, and the deployed model remains RGB-only. On the 4B backbone, GPD achieves 57.1 on VSI-Bench and 37.6 average across MindCube, SPARBench, MMSI-Bench, and ViewSpatial, outperforming both GRPO and answer-privileged OPSD across spatial reasoning benchmarks. Ablations confirm the complementarity of 3D and answer privilege, the advantage of question-conditioned routing over full-context injection, and the benefit of restricting distillation to incorrect trajectories.