R1-SyntheticVL: Is Synthetic Data from Generative Models Ready for Multimodal Large Language Model?

AI summary

Overview

• Research area: Multimodal data synthesis and post-training for Multimodal Large Language Models (MLLMs), specifically generating synthetic image-text question-answer data with text-to-image generative models and using it for reinforcement learning.
• Technical level: Advanced. The paper assumes familiarity with MLLM training pipelines, Chain-of-Thought (CoT) data, GRPO reinforcement learning, and text-to-image generation models.
• Scope: The paper proposes Collective Adversarial Data Synthesis (CADS), uses it to build a 20K-instance synthetic dataset (MMSynthetic-20K), and trains a model (R1-SyntheticVL) that is benchmarked against open-source and closed-source MLLMs.

What This Paper Is About

MLLMs need large amounts of high-quality multimodal training data, but such data is scarce, expensive to annotate, and especially hard to obtain for complex real-world tasks that require reasoning and CoT annotations. The authors ask whether synthetic data from modern generative models is actually ready to fill this gap, and they find that directly prompting a strong generator such as Nano Banana Pro produces data that is low quality, repetitive, and too easy. Their goal is to build a general synthesis framework that reliably produces high-quality, diverse, and challenging multimodal data, and to verify it by training an MLLM with reinforcement learning on the resulting dataset.

Key Contributions

1. Collective Adversarial Data Synthesis (CADS): A general approach that uses collective intelligence from multiple MLLMs plus adversarial learning to synthesize high-quality, diverse, and challenging multimodal data. The authors state this is the first work exploring synthetic multimodal data from generative models for MLLMs and the first to introduce collective adversarial learning into MLLM data synthesis.
2. MMSynthetic-20K: A dataset of 20K high-quality synthesized multimodal instances, each containing a visual input, a textual instruction, and an answer, intended as a resource for mitigating data scarcity in MLLM training.
3. R1-SyntheticVL: A 7B MLLM trained from Qwen2.5-VL-7B with GRPO reinforcement learning on the synthetic data only, claimed to show strong performance on complex real-world multimodal tasks.
4. Extensive experiments: Ablations, comparisons against real data, complementarity experiments, and a data-scaling study validating the framework and the dataset.

Main Findings

• Direct generator prompting is weak, but the right generator helps: On MathVista, fine-tuning results are 68.2 for the Qwen2.5-VL-7B baseline, 66.3 with Stable Diffusion data, 67.9 with Nano Banana data, 70.8 with Nano Banana Pro data, and 75.6 with CADS. Stable Diffusion and Nano Banana degrade performance below the baseline, while Nano Banana Pro yields a gain.
• Every CADS component adds value (ablation, MathVista): baseline 68.2; plus Nano Banana Pro 70.8; plus CAD-Generate and CAD-Judge 73.0; plus CAD-Generate, CAD-Judge, and Adversarial Context Optimization 74.6; full CADS 75.6.
• Strong results on reasoning benchmarks: R1-SyntheticVL reaches 75.6 on MathVista, surpassing ThinkLite-VL-7B (75.1) and Vision-R1-7B (73.5). On MMMU-Pro it scores 42.0 and the authors report it performs best among compared models, including GPT-4o (54.0 on MMMU-Pro as listed) and Claude-3.5-Sonnet. The paper states R1-SyntheticVL achieves the best performance on the majority of the 6 evaluated benchmarks.
• Synthetic data rivals real data: With 2,000 training instances each, real data yields 72.2 on MathVista while MMSynthetic yields 73.3 under identical GRPO configurations.
• Synthetic and real data are complementary: Combining 2K real and 2K synthetic instances (4K total) gives 74.6, the best result in that comparison.
• Performance scales with synthetic data volume: Training with 0.5k, 2k, 10k, and 20k instances improves MathVista accuracy steadily; at 20k the accuracy improves by 7.4% over the baseline, and the gain continues from 10k to 20k, indicating the model has not saturated.
• Synthesized samples cover broad disciplines: Qualitative illustrations span Mathematics, Biology, Chemistry, Physics, and Chart Understanding, with visual content such as geometric solids, circuit diagrams, and topographic maps that reflect the textual constraints.

Methodology in Plain English

The framework runs in two cyclic phases with an optimization loop between them.

CAD-Generate (generation). Starting from seed data that can be either an existing multimodal sample or a plain text description of a target task, a group of MLLMs works collectively through three steps: (1) rationale analysis, where each model extracts the knowledge domain and the underlying reasoning logic of the seed; (2) synthesis strategy generation, where four meta-strategies are adapted per seed into fine-grained strategies — Numerical & Parameter Variation (change quantities while keeping structure), Logic Reversion (swap conditions and target), Auxiliary Extension (add a reinforcing element such as an altitude or median), and Isomorphic Scenario Transfer (map the same logic into a different visual scenario, e.g., from billiard balls to vehicles); and (3) visual prompt generation, where the new problem is turned into a detailed prompt specifying spatial layout, object attributes, data values, and geometric relationships so that Nano Banana Pro produces an image aligned with the text.

CAD-Judge (judgment). Multiple MLLMs act as judges and each tries to solve the synthesized question. A consensus score C counts how many judges match the generated ground truth. Instances where C = 0 (nobody can solve it, suggesting ambiguity or errors) are filtered out.

Adversarial Context Optimization. Instances where 1 ≤ C < K (some judges solve it, others do not) are treated as high-value adversarial instances lying on the models' decision boundaries — neither trivial unanimous cases (C = K) nor noise (C = 0). The framework "reflects" on the judges' disagreements to distill insights about successes and errors, then "optimizes" the generation context with those insights so that subsequent rounds produce more challenging, higher-value data.

Training setup. Data synthesis uses four models — GPT-4o, Gemini-2.5-Flash, DeepSeek-R1, and Claude-4-Sonnet — with a maximum of 10 generation iterations per seed. Training uses Qwen2.5-VL-7B as the base model with GRPO on the EasyR1 codebase, on 8 NVIDIA H20 GPUs, with 8 samples per question, global batch size 128, rollout batch size 256, rollout temperature 1.0, KL penalty coefficient 1e-2, learning rate 1e-6, top p = 1, top k = -1, and a reward combining accuracy and format rewards.

Why This Matters

The work targets the data bottleneck that limits further MLLM progress, especially for domain-specific or reasoning-heavy data that is scarce and costly to annotate. If synthetic data can match or complement real data, it changes the economics of building capable multimodal systems.

Real-world applications (as framed by the paper's focus on complex real-world multimodal tasks):

• Education and tutoring systems that need large volumes of graded, diagram-based problems across Mathematics, Physics, Chemistry, and Biology.
• Scientific and technical reasoning assistants that must interpret figures such as circuit diagrams, geometric constructions, and topographic maps.
• Chart and document understanding tools, a domain the paper explicitly evaluates through the CharXiv benchmark.
• Domain-specific and safety-sensitive settings, which the paper identifies as inherently scarce and hard to obtain (for example, medical data), where synthesis reduces reliance on collecting and annotating sensitive real data.

Industry relevance: The paper emphasizes cost-efficiency and scalability — automated synthesis reduces dependence on expensive, labor-intensive manual collection and annotation, offering a resource-efficient path to training high-performing MLLMs. The released code, model, and data (announced at a public GitHub repository) lower the barrier for teams without large annotation budgets, and the results suggest synthetic data can be a substitute or an additive component to existing real-data pipelines. The work was supported by the Tencent WeChat Rhino-Bird Focused Research Program and a PolyU Internal Fund.

Future Directions

• Scaling beyond 20K instances: The scaling study shows gains continuing from 10k to 20k without saturation, so the behavior at substantially larger scales — and the point at which returns diminish — remains open.
• Broadening the collective and the generator: The experiments use four specific models for generation and judging and rely on Nano Banana Pro for visuals; whether the framework transfers to other or open-source generators, and how sensitive results are to the choice and number of collective members, is not established here.
• Extending beyond static image-text pairs: The current dataset covers text plus a single generated image; extending CADS to other modalities and to data derived from scratch rather than from seed data is a natural next step.
• Cost and efficiency of synthesis: With up to 10 iterations per seed and multiple MLLM judges and generators per instance, the compute and API cost profile of CADS, and how to reduce it, is a practical question the paper does not quantify.

Target Audience

Researchers and engineers working on MLLM training, multimodal data synthesis, and reinforcement-learning post-training, particularly those interested in reducing dependence on manually annotated data. It is also useful for practitioners who need to generate reasoning-heavy visual question-answer data for specialized domains, and for readers tracking whether text-to-image generative models are mature enough to serve as training-data sources rather than just content-creation tools.

Authors’ abstract

In this work, we aim to develop effective data synthesis techniques that autonomously synthesize multimodal training data for enhancing MLLMs in solving complex real-world tasks. To this end, we propose Collective Adversarial Data Synthesis (CADS), a novel and general approach to synthesize high-quality, diverse and challenging multimodal data for MLLMs. The core idea of CADS is to leverage collective intelligence to ensure high-quality and diverse generation, while exploring adversarial learning to synthesize challenging samples for effectively driving model improvement. Specifically, CADS operates with two cyclic phases, i.e., Collective Adversarial Data Generation (CAD-Generate) and Collective Adversarial Data Judgment (CAD-Judge). CAD-Generate leverages collective knowledge to jointly generate new and diverse multimodal data, while CAD-Judge collaboratively assesses the quality of synthesized data. In addition, CADS introduces an Adversarial Context Optimization mechanism to optimize the generation context to encourage challenging and high-value data generation. With CADS, we construct MMSynthetic-20K and train our model R1-SyntheticVL, which demonstrates superior performance on various benchmarks.