MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoning

AI summary

Overview

Research area: Multimodal reasoning — specifically, how to build training data and post-training pipelines for vision-language models (VLMs) that reason over images.

Technical level: Advanced. The paper assumes working knowledge of supervised fine-tuning (SFT), reinforcement learning for LLMs (GRPO/GSPO-style policy optimization), and knowledge distillation, and it reasons about KL-divergence objectives and teacher-student distributions.

Scope: A recipe paper — covering open-data cleaning, SFT and RL dataset construction, expert specialization via RL, and multi-teacher on-policy distillation — evaluated on 15 multimodal benchmarks at the 4B and 9B parameter scales.

What This Paper Is About

Open multimodal reasoning models have improved mainly by scaling up reasoning supervision, but the paper argues the real bottleneck is the post-training recipe: uneven data quality, expensive and inefficient supervision construction, imbalanced difficulty, and interference when different reasoning domains are trained together. The authors build MMVistaReason (MVR), a three-part framework that broadens task coverage (Analytical plus Real-World reasoning), constructs two compact datasets (MVR-SFT-528K and MVR-RL-63K) through cascaded teacher distillation and frontier filtering, and then trains domain-specialized RL experts that are merged back into one generalist via multi-teacher on-policy distillation (MOPD).

Key Contributions

1. A broader, balanced task taxonomy and recipe suite. MMVistaReason splits multimodal reasoning into two complementary groups — Analytical (mathematics, science, visual logic, charts; structured and knowledge-intensive) and Real-World (general perception, counting, spatial understanding, GUI grounding) — and shows MVR-4B beating MMFineReason-8B at half the model size with roughly 70% fewer SFT samples.

2. An efficient SFT and RL data construction pipeline. The pipeline combines staged rule-based plus MLLM-assisted cleaning, a difficulty-aware cascaded teacher distillation (Qwen3.5-4B as a difficulty filter, then Qwen3.5-9B → 27B → 122B as progressively stronger teachers), Answer-Likelihood Best-of-N trajectory selection, and scale-specific frontier filtering, yielding MVR-SFT-528K (527,809 samples, 1.04B tokens) and MVR-RL-63K.

3. A "specialize-then-integrate" training recipe. Complementary RL experts are trained per reasoning group with GSPO, then consolidated into a unified student through multi-teacher on-policy distillation, with both forward-KL and reverse-KL objectives studied. The authors report this reduces mixed-domain interference relative to training a single model on mixed-domain RL.

4. A capacity-dependent analysis of supervision. The paper characterizes how supervision difficulty, data scale, and trajectory quality interact with student model size, showing that smaller students benefit from broader difficulty coverage while a 9B student matches full-data performance with only a 124K moderately difficult subset.

Main Findings

• Strong benchmark averages at compact scale. Across 15 multimodal benchmarks, MVR-4B scores an overall average of 72.8, outperforming MMFineReason-8B (70.9) by 1.9 points and Qwen3.5-9B (Instruct) (72.1) by 0.7 points. MVR-9B reaches 74.4, surpassing Qwen3.5-35B-A3B (Instruct) (74.1) by 0.3 points and approaching GPT-5.1 (74.6).

• Real-World reasoning is the largest relative gain. MVR-4B achieves a Real-World average of 82.0, +3.3 over MFR-8B, with OSWorld-G at 58.5 (+10.2) and CountBenchQA at 91.5 (+3.4). MVR-9B reaches 83.8, 1.1 points above Qwen3.5-35B-A3B (Instruct), with OSWorld-G at 64.8 and CountBenchQA at 92.8.

• Analytical reasoning stays competitive. MVR-4B posts an Analytical average of 66.6, +1.0 over Qwen3.5-9B (Instruct) and +0.8 over MFR-8B; MVR-9B reaches 68.1. MVR-9B scores 82.4 on MathVerse mini (above the 35B model and GPT-5.1) and 87.6 on ChartQA.

• Each stage adds measurable value. SFT lifts the overall average from 66.6 to 70.8 at 4B and from 68.9 to 72.3 at 9B, both above the corresponding Qwen3.5-Instruct models (70.4 and 72.1). MOPD then raises these to 72.8 and 74.4, with Real-World averages improving 79.7 → 82.0 and 81.6 → 83.8, and Analytical averages 64.9 → 66.6 and 66.1 → 68.1.

• Biggest per-benchmark gains come from integration. Over the SFT models, MOPD adds +4.5/+4.9 on OSWorld-G, +3.5/+2.2 on RealWorldQA, and +3.3/+3.2 on CharXiv reas. at 4B/9B.

• Data scale alone does not predict performance. Fine-tuning Qwen3.5-2B-Base, 4B-Base, and 9B-Base on MVR-SFT-49K, 124K, 355K, and the full 528K shows the 2B model improving from 63.9 (49K) to 65.6 (528K) and the 4B from 69.8 to 70.8, while the 9B model scores 72.3 on the 124K subset — matching full 528K while beating the 355K subset (71.7) and Qwen3.5-9B (Instruct) (72.1).

• Supervision difficulty must match capacity. The 49K subset, distilled from the strongest 122B teacher, contains the hardest samples yet performs worst for the 2B and 4B students. The 355K subset, produced by a teacher of the same 9B scale, gives only limited gains to the 9B student.

• Trajectory selection matters. Answer-Likelihood Best-of-N — ranking verified-correct trajectories by the length-normalized log-likelihood of the final answer tokens under the teacher — is reported as more reliable than random or length-based selection across model scales (the truncated text covers this comparison at the figure level).

• Mixed-domain RL causes negative transfer; MOPD does not. Training one model on mixed-domain RL produces benchmark-level negative transfer, whereas multi-teacher on-policy distillation gives more consistent capability integration. The preferred KL direction (forward vs. reverse) varies with model scale.

• Domain composition of the SFT corpus is complementary, not replicated. MVR-SFT-355K supplies 64.1–73.2% of Science, Mathematics, Logic/Game/Puzzle, and Chart/Table/Doc; MVR-SFT-49K emphasizes GUI/Web (38.3%); MVR-SFT-124K contributes relatively more Spatial data (27.0%). Science, Mathematics, and Chart/Table/Doc together make up 78.61% of the corpus.

• RL data is genuinely frontier data. 80.9% of MVR-RL-63K comes from the three SFT-derived subsets and 19.1% from Teacher-Unsolved samples; 83.83% of RL samples fall in the Post-SFT Mastery range of 0.375–0.625 (pass rate over eight rollouts), i.e. solvable but not yet mastered.

Methodology in Plain English

Step 1 — Clean the raw data. The authors pool VQA-style open datasets and remove duplicates using image hashes (SHA-256), perceptual hashes (pHash), and CLIP-based semantic similarity; they discard corrupted, low-resolution, blank, blurry, or extreme-aspect-ratio images. A strong model, Qwen3.5-122B-A10B, then standardizes language, strips ads/watermarks/links, drops unanswerable items, rewrites unhelpful meta-instructions, extracts answers from noisy fields, and canonicalizes them (for example, recovering option letters for multiple-choice questions). Every sample gets a reasoning-domain and question-type label.

Step 2 — Build SFT data by escalating only when needed. Running the biggest teacher on every sample is expensive, so the pipeline first asks Qwen3.5-4B (Instruct) four times; anything it solves is dropped as too easy. Remaining hard samples go to Qwen3.5-9B, and only those it fails go to 27B, then 122B, with four attempts each. Samples no teacher solves are discarded as too hard, ambiguous, or noisy. When several correct trajectories exist, the one whose final answer is most confidently predicted by the teacher (length-normalized log-likelihood) is kept.

Step 3 — Build RL data from the student's own frontier. For each model scale, the SFT model is run eight times per candidate; samples solved one to seven times are kept (with priority on two to six). Fully failed or fully solved samples are excluded. The kept prompts are balanced across the two reasoning groups.

Step 4 — Train specialists, then merge them. A single SFT model is trained on all 528K samples. Two RL experts are then trained with GSPO (global batch size 256, 8 rollouts per example) — one on Real-World data, one on Analytical data, with Qwen3.5-27B acting as the answer-correctness judge for rewards. Finally, MOPD has the SFT generalist generate its own responses, routes each sample to the matching expert, and asks that expert to score the top-32 tokens at every prefix the student actually visited. The student is trained to match that distribution via forward or reverse KL. Training uses LLaMA-Factory, VeRL, and VLMEvalKit, with greedy decoding and a maximum generation length of 16,384 tokens.

What makes it distinctive: the cascade means expensive teachers are only invoked on what cheaper teachers cannot solve; the "specialize-then-integrate" step deliberately avoids mixing incompatible domains in one RL objective; and the analysis treats data difficulty as something to be matched to student capacity rather than maximized.

Why This Matters

Impact on research. The paper moves the conversation from "more reasoning tokens" to recipe design: how supervision difficulty should be matched to model capacity, why mixed-domain RL can hurt specific benchmarks, and how on-policy distillation can merge specialists without that penalty. Its finding that a 124K subset matches a 528K mixture at 9B is a concrete argument that difficulty stratification, not volume, drives returns. The release of MVR-SFT-528K and MVR-RL-63K as open resources, plus a public repository, supports reproducibility in an area where data pipelines are often undisclosed.

Real-world applications:

• GUI and computer-use agents. The jump on OSWorld-G (58.5 at 4B, 64.8 at 9B) makes compact models more viable for on-device or low-cost interface automation.
• Assistive visual question answering. Gains on RealWorldQA, CV-Bench-2D/3D, and CountBenchQA translate to perception and counting tasks used in accessibility tools and scene understanding.
• Document, chart, and diagram analysis. ChartQA 87.6 and CharXiv reas. 68.8 at 9B support automated report, dashboard, and scientific-figure interpretation.
• Cost-sensitive deployment in education and technical support. A 4B model that beats several 8B–9B competitors on science and math benchmarks lowers the hardware bar for tutoring and step-by-step problem-solving assistants.

Industry relevance. The efficiency claims are the headline for practitioners: about 70% fewer SFT samples than MMFineReason at half the model size, plus a cascaded teacher scheme that avoids running a 122B model on every sample. It is directly relevant to teams at Ant Group's scale (a co-affiliated author group) that need compact, reliably trained multimodal models rather than frontier-scale ones.

Future Directions

• Extending beyond two reasoning groups. The taxonomy covers Analytical and Real-World; adding groups such as video, multi-image, or agentic/interactive reasoning would test whether the specialize-then-integrate recipe generalizes.
• A principled rule for KL direction selection. The paper reports that the preferred KL direction (forward vs. reverse) varies with model scale but does not give a predictive rule; identifying one — or an adaptive combination — is a natural next step.
• Better handling of the Teacher-Unsolved pool. 19.1% of the RL set comes from samples no teacher solved; whether these can be converted into supervision through stronger teachers, verifier-guided search, or different reward shaping remains open.
• Difficulty scheduling during training. The capacity-dependent findings are currently used for static subset selection; curriculum-style approaches that shift supervision difficulty during SFT or RL could exploit the same effect more directly.

Target Audience

Researchers and engineers working on multimodal post-training who already understand SFT and RL for language models and want a concrete, end-to-end recipe rather than a single architectural trick. It is especially useful for teams building compact (2B–9B) VLMs under compute constraints, for data-curation groups designing distillation pipelines, and for readers interested in multi-teacher on-policy distillation and cross-domain transfer in vision-language models. Readers looking for a first introduction to multimodal reasoning will find the data pipeline accessible but the training sections demanding.

Authors’ abstract

Open multimodal reasoning models have benefited from large-scale reasoning supervision, yet reliable post-training remains challenging due to uneven data quality, inefficient supervision construction, imbalanced difficulty, and cross-domain interference. We introduce MMVistaReason (MVR), an open-data post-training recipe with three components: (1) broader capability coverage across complementary Analytical and Real-World reasoning groups, emphasizing structured reasoning versus visual perception and spatial grounding; (2) efficient SFT and RL data construction, standardizing heterogeneous open data through staged cleaning and annotation, combining difficulty-aware cascaded teacher distillation with answer-likelihood-based trajectory selection to construct MVR-SFT-528K, and applying scale-specific frontier filtering for MVR-RL-63K; and (3) specialize-then-integrate training, which trains complementary RL experts and consolidates their capabilities through multi-teacher on-policy distillation (MOPD). Our analyses reveal a capacity-dependent interaction between supervision difficulty, trajectory quality, and model capacity: smaller students benefit more from selected supervision, while larger students are robust to trajectory variation and mixed-domain interference. Mixed-domain RL introduces benchmark-level negative transfer, whereas MOPD provides consistent capability integration, with the preferred KL direction varying across model scales. Across 15 multimodal benchmarks, MVR-4B achieves an average score of 72.8, outperforming Qwen3.5-9B (Instruct) and MMFineReason-8B while using about 70% fewer samples than MMFineReason. Scaling to 9B improves the average to 74.4, surpassing Qwen3.5-35B-A3B (Instruct). Overall, MMVistaReason demonstrates that systematic open-data construction and capacity-aware post-training provide a practical and scalable path toward reliable multimodal reasoning.