Zooming without Zooming v2: From Seeing Details to Connecting Evidence for Generalizable Multimodal Perception

AI summary

Overview

Research area: Computer Vision / multimodal large language models — specifically visual perception training data synthesis and reinforcement learning for vision-language models (VLMs).

Technical level: Intermediate. Readers will get the most from this paper with some familiarity with vision-language models, reinforcement learning with verifiable rewards (RLVR), and benchmark evaluation, though the central idea is explained in plain terms.

Scope: The paper introduces EvidenceConnect, an automated pipeline that synthesizes multi-region, single-image questions requiring models to "connect evidence," and shows that RLVR training on this data (producing models named ZwZ2) improves performance on 25 benchmarks spanning fine-grained perception, spatial perception, long-context understanding, embodied spatial reasoning, multimodal agents, and general visual reasoning.

What This Paper Is About

Multimodal models are good at spotting details in a single region of an image, but they struggle when the answer requires piecing together facts scattered across objects, relationships, frames, or dialogue turns — the kind of skill needed for spatial perception (which underpins embodied AI) and long-context understanding (which underpins multimodal agents). Building high-quality training data for those two tasks is expensive and error-prone, because reliable labels require precise localization across multiple objects, or long, coherent multi-turn/multi-image sequences.

The paper's core claim is that these seemingly different tasks share one underlying capability — connecting evidence: identifying relevant visual facts, binding each fact to the correct entity, recovering relations among them, and composing them into an answer. Since that capability is much easier to synthesize training data for than the downstream tasks themselves, the authors build a data generator targeting it and test whether the resulting models generalize.

Key Contributions

1. A named capability as a training target. The authors identify "connecting evidence" as an atomic VLM capability shared by fine-grained perception, spatial perception, and multimodal long-context understanding, and argue it is a far cheaper supervision target than the end tasks.

2. EvidenceConnect, a fully automated synthesis pipeline. It builds questions over coherent groups of 3–7 regions within a single image, generated from privileged zoomed layouts with colored region outlines, covering three question forms: aggregation (counts/numeric facts, possibly with short calculation), comparison and matching (over a closed candidate set), and dependency chains/compositions.

3. Two-stage verification without human annotation. Cross-model answer anchoring keeps a reference answer only when two VLMs from different model families agree by clear majority; adaptive drop-one testing masks one region at a time and rejects a question only when more than one region is found empirically redundant. Candidates also go through rule-based rejection (unused regions, broken reasoning certificates, leaked answers via outline colors, inconsistent arithmetic), multiple-choice conversion, and difficulty filtering.

4. Empirical demonstration of transfer. Training Qwen3.5-4B, Qwen3.5-9B, and Qwen3.6-27B with GRPO on 20.4K synthesized examples (no supervised fine-tuning, no benchmark-specific formats) yields ZwZ2-4B/9B/27B, which improve over their base models on all 25 benchmarks at every scale — including multi-image, interactive, and agent benchmarks the training data never covered.

Main Findings

• Consistent gains across all 25 benchmarks at all three scales. Overall averages rise from 49.41 to 54.79 (4B), 51.63 to 56.59 (9B), and 55.52 to 60.51 (27B), i.e., gains of 5.39, 4.96, and 4.99 points respectively.

• Fine-grained and spatial perception improve by 5.2–6.3 points on average, with gains of up to 11.1 points on MIMIC and 10.8 points on CountQA. Embodied spatial reasoning improves by 5.5–8.1 points, including 10.5 points on ERQA at 27B.

• Single-image training transfers to multi-image and interactive settings. Long-context understanding improves at every scale (most at 4B, where the domain average rises 5.6 points, including +10.0 on MileBench from 60.08 to 70.08). AgentVista improves by 4.8–6.2 points, and GUI grounding improves consistently.

• ZwZ2-27B beats the three frontier reference models on overall accuracy: 60.51% versus 54.02% (GPT-5.6-Luna), 57.24% (GLM-5.3-Flash), and 55.47% (DeepSeek-V4.1-Flash), and leads the fine-grained perception domain. The paper notes the advanced models' input resolution was constrained by platform limitations and all models were evaluated in low-thinking or non-thinking modes.

• It outperforms five prior synthetic datasets trained with the same GRPO recipe on Qwen3.5-4B. MMHops (31K), TVG (19.4K), F²RVLM (37K), and CRIT (30K) each improved MileBench but degraded HR-Bench, CountQA, MindCube-Tiny, and MemLens, and all four dropped the 11-benchmark average below the base model's 47.16. EvidenceConnect (20.4K) reaches 53.86 and wins on all 11 benchmarks against those four.

• The closest baseline is ZwZ (73K), which is 3.6× more data. EvidenceConnect reaches 53.86 versus 49.86 for ZwZ and wins on 10 of 11 benchmarks, trailing only MileBench by 0.13 points. The sharpest contrast is MindCube-Tiny, where ZwZ reduces accuracy by 13.8 points while EvidenceConnect improves it by 5.4 points — consistent with the claim that embodied spatial reasoning depends on connecting regions rather than inspecting one.

• Drop-one verification matters. At 7K examples, adding it raises the 11-benchmark average from 51.03 to 52.65 (+1.63), improving 10 of 11 benchmarks, with the largest gains on MindCube-Tiny (+5.15), O3-Bench (+4.64), and CountQA (+3.79); ERQA decreases by 4.00 points.

• Privileged views matter. Synthesis from privileged zoomed layouts scores 52.65 versus 51.38 from original images (+1.27), higher on 9 of 11 benchmarks; original-image synthesis is better on Chartography (+4.00) and ERQA (+4.75).

• Gains extend to other base models. Qwen3-VL-2B goes from 36.82 to 41.63 (+4.81) and Qwen3-VL-4B from 41.70 to 44.61 (+2.91) on the same 11 benchmarks.

• Data scale helps but saturates gradually. 7K examples already lift the average from 47.16 to 52.65 (+5.49); the full 20.4K adds a further 1.21 points to 53.86.

• Gains narrow at the longest contexts. On MemLens, gains persist up to 256K tokens, but at 128K on the MMLongBench visual core, ZwZ2-4B and ZwZ2-9B merely match their base models — consistent with training contexts being a single (short) image.

• Attention shifts toward evidence. On 405 TreeBench examples with annotated evidence boxes, ZwZ2 has higher mean top-k IoU with ground-truth evidence tokens in 7 of 8 full-attention layers at 4B and 6 of 8 at 9B.

Methodology in Plain English

The approach is to build a hard-to-fake practice task whose answer genuinely depends on combining several pieces of visual evidence, then let reinforcement learning do the rest.

Step 1 — Find good images and regions. Starting from SA-1B images, the pipeline keeps only detailed, high-resolution ones (longer side ≥1280 pixels, intensity entropy ≥6.5 bits, Canny edge density ≥0.04, luminance standard deviation ≥60). A strong VLM (Qwen3.5-397B-A17B) proposes up to 18 bounding boxes with short descriptions; boxes smaller than 0.04% or larger than 18% of the image are dropped, as are boxes overlapping an already-kept box with IoU ≥0.65. Images with fewer than four usable regions are skipped.

Step 2 — Group regions that can support one question. A text-only call proposes up to six groups of 3–7 regions, each tagged with one of 22 task families and a "necessity mode" describing how removing a region should change the answer. These map onto the three question forms (aggregation; comparison/matching; dependency chains).

Step 3 — Show the model a zoomed private view. For question writing, each group is rendered as a privileged layout: native-resolution crops with 8% margin, arranged on a compact canvas that preserves relative order, each region outlined in one of seven colors. This view is only used to build data — the trained model never sees it.

Step 4 — Write questions with an audit trail. The generator produces questions plus concise answers, states each region's contribution, predicts what the answer would be if that region were removed, and supplies a short step-by-step reasoning certificate. Deterministic rules reject broken or leaky candidates.

Step 5 — Check the answer and check the dependence. Two VLMs from different families answer the question from the privileged layout several times without seeing the proposed answer; a clear majority agreement fixes the reference answer. Then each region is masked in turn with a gray placeholder; the question is discarded only if more than one region turns out to be redundant, a tolerance designed for noise from overlapping crop margins and verifier variability.

Step 6 — Convert to the model's view and to multiple choice. Each verified question is paired with the full image with colored region outlines (and, for aggregate questions, sometimes rewritten with natural descriptions instead of color markers — accepted only if both anchoring families confirm unique references, an unchanged task, and the same answer). Four-option multiple choice enables rule-based rewards, and a lightweight VLM (Qwen3.6-35B-A3B) answers each example four times, with always-correct examples discarded as too easy.

Step 7 — Train with RLVR. GRPO in non-thinking mode, same data and configuration for all three scales, no supervised fine-tuning.

Why This Matters

Impact on research. The paper reframes data scarcity as a data-design problem: instead of annotating the expensive end task, find the shared sub-capability that is cheap to synthesize and verifiable. If connecting evidence really is a transferable target, many costly supervision problems (embodied trajectories, long multi-turn agent histories) could be approached through much cheaper proxies. The comparison table also shows a practical warning — training on other synthetic datasets improved one benchmark (MileBench) while degrading others and the overall average, which is a useful data point about narrow synthetic supervision.

Real-world applications:

• Embodied AI and robotics — spatial perception and viewpoint-dependent reasoning for navigation and manipulation, where the paper reports gains on ERQA, ViewSpatial-Bench, and MindCube-Tiny.
• GUI agents and computer-use assistants — improvements on ScreenSpot-Pro and UI-Vision grounding plus AgentVista suggest better element localization and multi-step agent task performance.
• Long-document and multi-image assistants — OCR, chart, and multi-image evidence integration (MMLongBench, MileBench, HR-Bench 4K/8K, Chartography) matter for document analysis and visual QA over reports or screenshots.
• Cost reduction in data operations — a fully automated pipeline with cross-model agreement checks and ablation-based quality control replaces human annotation for verification.

Industry relevance. The framework is released as code (inclusionAI/Zooming-without-Zooming-v2) with a Hugging Face collection, and it improves open base models at 4B, 9B, and 27B scales with the same data and recipe — precisely the range deployed in cost-sensitive production. ZwZ2-27B exceeding the reported overall scores of GPT-5.6-Luna, GLM-5.3-Flash, and DeepSeek-V4.1-Flash, while being trained on 20.4K synthesized examples with no SFT, is a direct argument for this style of post-training.

Future Directions

• Retrieval at very long contexts remains unsolved. Gains persist on MemLens to 256K tokens but flatten at 128K on the MMLongBench visual core; training data are single images, so the authors attribute the residual difficulty to retrieval from long inputs rather than evidence connection. Extending the pipeline to multi-image or multi-turn evidence groups is the obvious next test.
• The two verification steps are not uniformly beneficial. Drop-one verification costs 4.00 points on ERQA, and privileged views cost 4.00 on Chartography and 4.75 on ERQA. Understanding when each step helps — and whether the "more than one redundant region" tolerance is optimal — is open.
• Headroom against frontier references is uneven. ZwZ2-27B leads overall and on fine-grained perception, but trails on several domains (for example BabyVision at 28.35 versus 45.88 for DeepSeek-V4.1-Flash and 37.11 for GLM-5.3-Flash, and the agent domain where GLM reports 33.66 versus 51.81 for ZwZ2-27B but 21.53–22.01 on AgentVista for the frontier models), which raises questions about which capabilities the proxy task does not reach.
• Scaling the synthesized data and the capability definition. Going from 7K to 20.4K examples added only 1.21 points; whether larger or more diverse (multi-image, video, interactive) evidence groups continue to pay off, and whether "connecting evidence" can be decomposed into separately trainable sub-skills, are unanswered.

Target Audience

Researchers and engineers working on vision-language model post-training, multimodal data synthesis, and RLVR who want a cheap, automatable supervision signal with broad transfer. It is also relevant to practitioners building embodied AI, GUI agents, or long-context multimodal assistants, and to anyone designing synthetic-data pipelines who can borrow the two-stage verification pattern (cross-model answer agreement plus region ablation) independently of this specific task.

Authors’ abstract

Multimodal perception forms the foundation of multimodal intelligence. In particular, spatial perception and multimodal long-context understanding are two pivotal tasks that underpin embodied AI and multimodal agents. However, synthesizing targeted, high-quality training data for these tasks remains difficult and costly. Our key insight is that these tasks share an underlying capability, which we call connecting evidence: identifying relevant visual facts, binding them to the correct entities, recovering their relations, and composing them into an answer. Crucially, targeted training data are far easier to synthesize for this capability than for these tasks themselves. To strengthen this capability, we propose EvidenceConnect, an easy-to-implement data synthesis framework that uses zoomed views of related regions within a single image to generate questions requiring comparison, aggregation, or chaining of visual evidence. To ensure quality, a reference answer is retained only when independent VLMs agree on it, and region-ablation tests discard questions whose answer survives the removal of selected regions. We then train base VLMs on the resulting data with RLVR, obtaining Zooming without Zooming v2 (ZwZ2). Although the synthesized data are not tailored to any downstream benchmark, ZwZ2 improves fine-grained and spatial perception, with further gains on embodied spatial reasoning, multimodal long-context understanding, multimodal agent tasks, and general visual reasoning. These results establish EvidenceConnect as an effective framework for supervising evidence connection and suggest that strengthening this shared capability leads to more generalizable multimodal perception.