Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention

AI summary

Overview

Research area: Multimodal large language models (MLLMs), specifically the interpretability of vision–language fusion and training-free methods for improving visual grounding.

Technical level: Advanced.

Scope: The paper performs layer-wise masking and attention analyses across seven MLLMs and six multimodal benchmarks to locate where visual information is fused into language representations, then proposes a training-free contrastive attention method that uses those findings to refine the visual input.

What This Paper Is About

MLLMs can answer questions about images, but it is unclear at which internal layers the visual and textual streams actually merge. The authors first probe this by masking image tokens at every layer and measuring the damage to accuracy, plus by tracking how attention over image regions evolves with depth. They then convert those observations into a training-free method that contrasts the attention of an early "pre-integrated" layer against the final layer, crops the image to the regions whose attention changed most, and re-runs the model on the original plus cropped image.

Key Contributions

1. A comprehensive layer-wise investigation of visual fusion in MLLMs. Systematic visual masking across seven representative models identifies key fusion layers, reveals a consistent shallow-to-middle fusion pattern, and uncovers a "review" phenomenon in some architectures.
2. The identification of persistent high-attention noise: irrelevant image regions receive strong attention across many layers, not just the final one, and this bias propagates rather than being reformed at each layer.
3. A training-free contrastive attention framework that models the attention transformation between a pre-integrated layer and the final layer, suppressing shared noise and highlighting text-guided attention shifts.
4. Extensive experiments across multiple MLLMs and multimodal benchmarks showing consistent improvements over prior training-free methods, including CD, DoLA, OPERA, and ViCrop, without additional training.

Main Findings

• Fusion is localized, not uniform: Across all seven models and the tested datasets, masking visual tokens causes the largest accuracy drops at a few specific layers. Figure 6 shows the fusion-layer distributions of the seven MLLMs are highly similar despite differences in architecture and training.
• Two-stage pattern: At shallow layers, zeroing image-token features causes a sharp performance decline, meaning visual input has not yet been integrated. Beyond layer 18–20, accuracy stabilizes even when visual input is masked, indicating fusion is largely complete and the model has shifted to text-dominant reasoning.
• A "review" phenomenon in deeper layers: In LLaVA-1.5, LLaVA-Next, InstructBLIP, and VIP-LLaVA, masking visual features at a deep layer again produces a noticeable accuracy drop, suggesting a secondary fusion pass. The authors describe this as the model "glancing back" at the image before answering.
• No review in Qwen models: Qwen2VL and Qwen2.5VL maintain small, stable Hellinger distances after layer 24 with no sign of late-stage re-fusion; Qwen2VL, Qwen2.5VL, and LLaVA-Onevision are the models without the review effect.
• Hellinger distance confirms progressive fusion: The distance between each layer's attention map and the final layer's map is large in shallow layers and decreases with depth, with fusion essentially finished before roughly layer 20. In LLaVA-1.5 and LLaVA-Next the distance rises again at layer 29, matching the review behavior.
• Persistent high-attention noise: In the LLaVA-1.5 example (question: "What type of plane is this?", key cue: the "LAPE" logo), attention over the correct region strengthens and peaks near layer 20, but irrelevant regions such as the tail stay strongly activated across all layers. Quantitative verification used 1,000 randomly selected TextVQA samples with manually annotated key and noise regions, averaged over all 32 layers of LLaVA-1.5: the target curve rises with depth while the noise curve stays consistently high.
• Early outputs are partly correct: On Qwen2.5VL for VQA, accuracy from intermediate-layer logits is nearly zero before layer 18 and rises notably after layer 18, showing that visually grounded answering becomes possible once fusion completes. Outputs produced as early as layer 18 can still be partially correct, improving with deeper output layers.
• Selected pre-integrated layers cluster at layer 2: Under an unconstrained selection range, layer 2 is chosen with overwhelmingly high frequency across datasets; in DocVQA, over 86% of samples select layer 2. Constraining the candidate set to the empirically identified fusion layers gives the best performance, while restricting to deep layers (after 17) performs poorly and the unconstrained "all layers" setting introduces noisy attention. Shallow layers (before 16) perform better than deep or all layers but worse than the fusion-layer strategy.
• Main results: On LLaVA, the method averages 59.16 versus 58.11 for OPERA, 57.60 for ViCrop, 57.36 for CD, 57.10 for DoLA, and 56.34 for the base model. On LLaVA-Next it averages 68.65 versus 67.73 (OPERA), 67.40 (ViCrop), 66.42 (DoLA), 66.22 (base), and 65.62 (CD). On Qwen2.5VL it averages 77.66 versus 77.29 (ViCrop), 76.89 (OPERA), 75.84 (DoLA), 75.83 (CD), and 75.15 (base).
• Inference cost: On GQA with an RTX A6000 GPU, the paper reports that CLIP and ViCrop achieve slightly faster inference but inferior overall performance; the added cost of computing contrastive attention is described as minimal.

Methodology in Plain English

The work has two halves.

Diagnosis. Image tokens occupy fixed positions in the sequence of transformer layers, so the authors can set those features to zero at any chosen layer without disturbing the rest of the computation. They do this at every layer and record how much accuracy falls. A big drop means that layer is where the model is still relying on visual evidence; a small drop means the model has already absorbed what it needs. They run this on VQAv2, GQA, TextVQA, OKVQA, VizWiz, and DocVQA across seven MLLMs. In parallel they track attention over image regions layer by layer, using the Hellinger distance to measure how far each layer's attention distribution sits from the final layer's.

Treatment. The key insight is that noise appears in early layers and simply carries forward, so subtracting one attention map from another cancels the shared noise. They pick a "pre-integrated" layer as the one with the largest Hellinger distance from the final layer (denoted layer k = 28 in the formalization), computed as H(P,Q) = (1/√2)·√(Σ_j(√p_j − √q_j)²), since that layer reflects early, task-agnostic perception and thus offers maximum contrast. The contrastive attention is the final-layer attention minus the pre-integrated-layer attention, IA = A^(k) − A^(i*). Regions with the largest positive change are where the model learned to look once text guidance kicked in. Those regions are used to segment the original image; the segmented image is then fed back together with the full original image for a second inference pass, reinforcing task-relevant cues while preserving global context. Nothing is trained. Unlike ViCrop, which fixes a crop layer per model (e.g., layer 14 in InstructBLIP or layer 17 in LLaVA) and needs model- and dataset-specific tuning, this selection adapts automatically.

Why This Matters

The paper reframes how visual information enters MLLMs: instead of treating attention as a single static map, it treats fusion as a layer-by-layer transition, and shows that a training-free intervention derived from that view beats existing decoding and cropping baselines. For research, it supplies a concrete, reproducible method for probing cross-modal fusion and shows that a "one layer fits all" interpretability assumption is wrong. It also identifies a failure mode—persistent high-attention noise—that is architectural rather than input-specific, which is useful for anyone studying hallucination or grounding errors.

Real-world applications suggested by the benchmarks and setup:

• Visual question answering assistants, where the improvement shows on GQA and VQAv2, and where re-attending to a cropped region mimics a human second glance.
• Document and screenshot understanding, where the largest single-model gains appear on DocVQA (68.10 for the method versus a 65.82 base on LLaVA-Next, and 68.10 average).
• Accessibility tools for blind and low-vision users, represented by VizWiz, where the method improves the average across LLaVA, LLaVA-Next, and Qwen2.5VL.
• Assistive or field use of text-heavy imagery, represented by TextVQA (in LLaVA-Next, 75.80 versus a 72.03 base).

Industry relevance: The method requires no retraining, no extra labels, and no architecture change, and it works across both the LLaVA and Qwen model families, so it can be dropped into existing deployed pipelines as a preprocessing and re-query step. It matters most where inference budget is already acceptable and where grounding errors are costly, such as document parsing, retail and industrial image inspection, and accessibility services.

Future Directions

• Explaining and exploiting the review phenomenon. The paper documents the late re-activation in LLaVA-1.5, LLaVA-Next, InstructBLIP, and VIP-LLaVA, and its absence in Qwen2VL, Qwen2.5VL, and LLaVA-Onevision, but does not explain what architectural or training difference causes it. Understanding this could make review layers an intentional design target.
• Making the pre-integrated layer selection more principled. The current criterion is maximal Hellinger distance from the final layer, which in unconstrained settings collapses onto layer 2 with over 86% frequency on DocVQA. Whether a dataset- or model-adaptive rule can beat the fusion-layer constraint is open.
• Reducing the second inference pass. The method re-inputs the original plus segmented image, adding computation. Whether the same contrastive signal can be injected without a second forward pass, or with cheaper segmentation, is unresolved.
• Explaining why the noise originates in early layers. The paper shows the bias emerges early and propagates rather than being reformed per layer; what generates it in the first place, and whether it can be prevented rather than subtracted, remains unaddressed.

Target Audience

Researchers and engineers working on multimodal LLM interpretability, vision-language grounding, and inference-time intervention methods. It is most useful to readers already comfortable with transformer internals, attention maps, and MLLM architectures such as LLaVA and Qwen-VL, and to practitioners who want a training-free accuracy improvement on visual question answering and document understanding without retraining a model. Readers looking for a first introduction to MLLMs would find the mechanistic analysis difficult without background.

Authors’ abstract

Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language understanding, yet how they internally integrate visual and textual information remains poorly understood. To bridge this gap, we perform a systematic layer-wise masking analysis across multiple architectures, revealing how visual-text fusion evolves within MLLMs. The results show that fusion emerges at several specific layers rather than being uniformly distributed across the network, and certain models exhibit a late-stage "review" phenomenon where visual signals are reactivated before output generation. Besides, we further analyze layer-wise attention evolution and observe persistent high-attention noise on irrelevant regions, along with gradually increasing attention on text-aligned areas. Guided by these insights, we introduce a training-free contrastive attention framework that models the transformation between early fusion and final layers to highlight meaningful attention shifts. Extensive experiments across various MLLMs and benchmarks validate our analysis and demonstrate that the proposed approach improves multimodal reasoning performance.