VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation

AI summary

Overview

Research area: Multimodal large language model (MLLM) training, specifically knowledge distillation for fine-grained visual perception, with connections to counterfactual visual supervision and on-policy distillation (OPD).

Technical level: Advanced. The paper assumes familiarity with token-level distillation objectives (Jensen–Shannon divergence, KL divergence), on-policy rollouts, privileged-information training, and log-probability geometry over a restricted candidate vocabulary.

Scope: The paper proposes Visual Attribution Distillation (VAD), a training-time method that estimates which parts of a privileged-teacher correction are actually supported by visual evidence, and reconstructs a student-anchored supervision target from that estimate, evaluated on six fine-grained visual benchmarks at 4B and 9B scale.

What This Paper Is About

Multimodal on-policy distillation trains a student model on its own generated text while a teacher that sees better visual evidence (a privileged view) supplies next-token corrections. The problem is that these corrections are "source-mixed": they blend genuine visual signal with the teacher's linguistic priors and teacher-specific quirks, so matching the full teacher distribution can teach the student the wrong thing. VAD's goal is to identify and use only the component of each correction that a controlled visual intervention actually explains.

Key Contributions

1. Diagnosis of source-mixed correction. The authors identify source-mixed teacher correction as a central limitation of privileged multimodal OPD, and show that direct teacher matching and positive visual-advantage weighting do not explicitly estimate which correction component is supported by a controlled visual intervention. A diagnostic reports that only 23.2% of the strongest response-token corrections for the 4B model and 22.8% for the 9B model are strongly aligned with the teacher's evidence-conditioned response.

2. The VAD algorithm. VAD uses a controlled visual-evidence intervention (evidence-present vs. evidence-removed view of the same image region, under the same fixed teacher and the same textual prefix) to construct a proxy direction, projects the privileged teacher correction onto it to obtain an aligned component and a proxy-unexplained residual, reconstructs a student-anchored target from the aligned component, separates support from refutation branches, and adds a weak teacher-correction regularizer for optimization stability.

3. Empirical validation across six benchmarks and two scales. VAD is shown to improve over direct privileged-view distillation and visual-advantage weighting at both 4B and 9B scale, with token-level semantic analyses showing the proxy-aligned component is enriched in task-relevant visual corrections and yields stronger target shifts, especially when evidence refutes a mistaken answer.

4. Controlled target-construction studies. Ablations compare six supervision targets under matched conditions, isolating gains from branch-aware target reconstruction and from the weak regularizer.

Main Findings

• Best scale-matched result at both scales. With matched data and update budgets on Qwen3.5-4B and Qwen3.5-9B, VAD reaches 78.32 and 79.93 on the unweighted six-benchmark mean (Avg₆), leading the best scale-matched alternative by 2.40 points at 4B and 2.80 points at 9B. Against the closely related Decomposed OPD baseline, gains are 2.95 and 2.88 points respectively.

• Cross-family comparison. The 4B VAD checkpoint exceeds Gemini 3 Flash (77.32), Gemini 3.1 Pro (78.04), and Qwen3.5-397B (76.49) on Avg₆; the 9B checkpoint reaches 79.93. The paper notes that architecture, data, and compute differences preclude controlled comparison for these rows.

• Broad rather than single-benchmark gains. VAD exceeds both Vision-OPD and Decomposed OPD on every benchmark at both scales. It leads every controlled post-training method on all six 4B evaluations; at 9B it leads five of six, with only ZoomBench favoring VA-OPD by 0.35 points.

• The decomposition separates semantics, not just magnitudes. In the relative top-5 composition, visual attributes, objects/content, and A–D decisions jointly account for 42.0% of the attributed direction r_vis, versus 26.7% in the complete correction and 17.7% in the residual. Language/format and artifact/meta semantics fall to 58.0% in r_vis, versus 73.3% in the complete correction and 82.3% in the residual.

• Token-level evidence supports semantic separation. Every A–D answer symbol and representative attribute/object tokens such as "vertical" and "metal" are more prominent in r_vis, while language scaffolding and artifact-related tokens such as "to" and "inside" are more prominent in the residual.

• Branch-aware target reconstruction is the main ablation gain. On Qwen3.5-4B, scalar shrinking improves direct matching by only 0.27 points in Avg₆ (75.92 to 76.19). Replacing the one-sided target with the branch-separated VAD target, without regularization, raises the average from 77.06 to 78.06, a 1.00-point gain. The weak anchor raises the one-sided and VAD targets to 77.52 and 78.32; full VAD is 0.80 points above regularized one-sided projection and 0.26 points above its target-only counterpart.

• Offline target effects. At fixed answer-slot states, correct-token support increases from 7.52 points for the teacher correction to 7.89 for the VAD target, while wrong-token suppression increases from 6.29 to 6.61 points; scalar shrinking and the one-sided target lie between these endpoints. The paper states the 95% confidence intervals overlap, so this supports a directional trend rather than pairwise statistical superiority.

• Held-out generalization does not collapse. On MMVP, CV-Bench, MMStar, and POPE (Avg₄), the 4B VAD model scores 79.14 versus a 4B base of 78.90, a change of +0.24, and is the best among the listed 4B post-training methods; Decomposed OPD is +0.05, while GRPO is −4.20, VA-OPD −2.72, V-Zero −0.66, and Vision-OPD −0.89. At 9B, the base is 82.39, with GRPO at 81.46 (−0.93), VA-OPD 79.20 (−3.18), V-Zero 80.53 (−1.86), and Vision-OPD 80.44 (−1.95); the 9B Decomposed OPD entry is cut off in the provided content.

• Visual-only supervision causes drift. When trained with the visual loss alone, the policy exhibited longer and more repetitive responses, delayed answer commitment, and unstable formatting and stopping — the stated motivation for adding the weak teacher regularizer.

Methodology in Plain English

VAD keeps the standard on-policy setup: the student generates a response from the full image, and the teacher is queried on the student's own prefixes. The twist is that the teacher is queried twice per position — once with a 2× crop containing the relevant evidence, and once with a degraded version of that same crop (0.1× bilinear downsampling followed by nearest-neighbor upsampling), keeping the text prefix identical. Because the teacher weights and the text are the same, the difference between these two distributions isolates how making the evidence visible changed the teacher's next-token preferences. The authors call this difference direction a proxy for the "visual evidence direction." It is signed: revealing evidence can raise the odds of some candidate tokens and lower others.

The complete teacher correction is then compared against this proxy. A one-sided projection extracts the part of the correction that points in the same direction as the intervention; whatever does not agree is set aside as a residual. The authors are explicit that this residual is a mixed leftover, not a claim that it is purely linguistic.

The actual training target is built by starting from the student's own current distribution on the full image and adding only the attributed visual shift, clipped within bounds. A refinement splits the proxy into positive and negative branches so support and refutation can be budgeted separately, with a cap on the support share to stop uncertain positive evidence from dominating. Supervision uses token-level Jensen–Shannon divergence — the same divergence as standard OPD, only with the target replaced.

Because visual-only supervision destabilized language behavior, a small regularizer compares the student against the original evidence-present teacher, weighted more heavily at positions where a smaller fraction of the correction was visually attributed. At inference, VAD is just the ordinary full-image student: no teacher, crop, or degraded view is used.

Training used 6,241 synthetic visual question-answering examples released by Vision-OPD, with no training image or question overlapping the six benchmarks. The student top-100 support plus a tail bucket was used, with batch size 96, 8 rollouts per prompt, learning rate 2×10⁻⁶, projection stabilizer ζ = 10⁻³, coordinate bound c = 20, and regularizer weight λ = 0.1; the positive-branch cap τ₊ was 0.8 for 4B and 0.7 for 9B. Evaluation covers VStar, ZoomBench, HRBench (4K/8K), and MME-RealWorld (EN/CN) using the official Vision-OPD inference and accuracy pipeline with GPT-OSS-120B as judge.

Why This Matters

Impact on research. The paper reframes multimodal OPD as an attribution problem rather than a weighting problem: instead of asking where or how strongly to distill, it asks which part of a teacher correction is warranted by controlled evidence. It also explicitly distinguishes itself from Decomposed OPD, which reconstructs visual supervision by prioritizing general visual-information matching, and from Anchored Residual OPD, which uses a future-conditioned textual residual. The residual is treated as mixed rather than as a clean linguistic signal, which is a more conservative claim than some prior decomposition work.

Real-world applications:

• Fine-grained visual inspection tasks such as reading small text, distinguishing materials and colors, or verifying spatial relations in product, document, or archival imagery.
• High-resolution perception settings where a small local detail determines the answer, such as medical, satellite, or industrial imagery review.
• Deploying capable visual assistants at small parameter counts (4B and 9B), which matters where inference cost or on-device execution is constrained.
• Real-world recognition benchmarks in both English and Chinese, relevant to multilingual deployment.

Industry relevance. The method adds no inference-time cost — the auxiliary teacher views exist only during training — so it is a drop-in improvement to an existing distillation pipeline rather than a new inference architecture. The reported parameter efficiency (a 4B model exceeding much larger open and closed systems on Avg₆) is directly relevant to teams trading model size against accuracy. Training code is released at github.com/DeepExperience/VAD_Multimodal_OPD, with models at huggingface.co/zhangkangning/VAD_for_Qwen3.5-4b and VAD_for_Qwen3.5-9b.

Future Directions

• Generalizing beyond the crop/degrade intervention. The evidence-removed view is a specific construction (0.1× downsampling followed by nearest-neighbor upsampling). Whether other intervention types produce equally useful proxies, and how sensitive the attribution is to that choice, is not resolved in the content provided.

• Extending beyond two scales and one student family. Results cover Qwen3.5-4B and Qwen3.5-9B. Scaling behavior above 9B, and behavior on other architectures or on students without a matched privileged-view teacher, are not reported.

• Quantifying the residual more sharply. The paper explicitly declines to claim that the proxy-unexplained residual is purely nonvisual. A method for further decomposing that residual — separating language priors from teacher-specific artifacts from degradation-related content — is a natural next step.

• Cost accounting and reproduction breadth. The paper defers wall-clock and GPU cost comparisons to an appendix (Appendix C); systematic efficiency accounting for the three-view training loop, plus ablation of the regularizer weight λ and the positive-branch cap τ₊, whose effects are shown in Figure 5 for 4B, remain open areas for follow-up.

Target Audience

This paper is most useful to researchers and engineers working on multimodal model post-training, knowledge distillation, and efficient small-model deployment — particularly those already using privileged-information or on-policy distillation pipelines. It will also interest readers studying counterfactual and contrastive visual supervision, and practitioners who need fine-grained visual accuracy at 4B–9B parameter scales with no added inference cost. A background in distillation objectives and autoregressive language modeling is assumed.

Authors’ abstract

Multimodal on-policy distillation (OPD) transfers fine-grained visual knowledge by supervising student-generated trajectories with a privileged-view teacher. Yet its next-token corrections are source-mixed, combining visual signals with linguistic priors and teacher-specific effects. The key challenge is to estimate which corrections are supported by visual evidence, not merely where or how strongly to distill. We introduce Visual Attribution Distillation (VAD), a counterfactual target-reconstruction algorithm that estimates the visually attributable part of a teacher correction. At each student-generated prefix, VAD evaluates the same fixed teacher with the relevant evidence present and removed. The corresponding change in centered log-probabilities defines ut, a signed proxy for the visual evidence direction that estimates how revealing the evidence supports or refutes candidate tokens. VAD projects the original correction onto this proxy to obtain an intervention-aligned component and a proxy-unexplained residual, then reconstructs a student-anchored target from the former. During training, this reconstructed target supplies the primary supervision signal, while the privileged teacher contributes a weak regularizer. Across six fine-grained visual benchmarks at 4B and 9B scales, VAD outperforms direct privileged-view distillation and visual-advantage weighting. Token- level and controlled-target analyses show that the proxy-aligned component is enriched in task-relevant visual corrections and yields stronger target shifts, especially when evidence refutes a mistaken answer. These results support counterfactual target reconstruction as an effective alternative to source-mixed supervision.