Mimic Human Cognition, Master Multi-Image Reasoning: A Meta-Action Framework for Enhanced Visual Understanding

AI summary

Overview

Research area: Multimodal large language model (MLLM) reasoning, specifically multi-image, multi-frame, and single-image visual understanding (arXiv category: Computer Vision).

Technical level: Advanced. The paper assumes familiarity with reinforcement learning for language models (GRPO, DAPO, KL and entropy regularization, pass@K evaluation) and with MLLM training pipelines (cold-start supervised fine-tuning followed by RL).

Scope: The paper proposes CINEMA, a cognition-inspired framework that decomposes multi-image reasoning into five structured "meta-actions" (Global, Focus, Hint, Think, Answer), trains a Qwen2.5VL 7B backbone on 56k cold-start and 58k RL instances, and evaluates on multi-image, video, and single-image benchmarks.

What This Paper Is About

Multimodal large language models perform well on single-image questions but degrade noticeably when a question requires reasoning over several images, where images may relate to each other semantically, spatially, or temporally and where the decisive evidence may sit in only one of many images. The paper's goal is to close that gap by imposing a human-like, staged reasoning structure on the model and by training it with a reinforcement learning scheme designed to keep reasoning strategies diverse rather than collapsing into a single deterministic pattern.

Key Contributions

1. A cognition-inspired meta-action framework (CINEMA) that decomposes multi-image reasoning into five ordered actions: Global (survey the whole question and inter-image relationships), Focus (concentrate on the key image and its salient details), Hint (summarize key points and error-prone or confusable information), Think (internal reasoning over gathered clues), and Answer (emit the final answer). For single-image inputs, the Global action is disallowed.

2. Retrieval-Based Tree Sampling for cold-start data, modeled on a student-teacher dynamic: a smaller model produces an initial trajectory, GPT-4o (as teacher) follows the student's action path and produces a correct trajectory, and a retrieval step then finds an alternative correct trajectory that is dissimilar from the first. Similarity is measured by encoding trajectories with BGE and computing cosine similarity. Each cold-start instance therefore carries two distinct correct trajectories.

3. A two-stage reinforcement learning paradigm for balancing exploration and exploitation. Stage one applies a Diversity-Preserving Strategy (DPS) built on DAPO with a trajectory homogeneity penalty that discourages repeated meta-action patterns within a sampling group; stage two applies DAPO with an annealing schedule to shift toward exploitation. KL-divergence and entropy regularization terms are omitted in the RL stage.

4. A constructed training corpus and broad evaluation: 56k cold-start instances (each with two reasoning trajectories) and 58k reinforcement learning instances spanning multi-image, multi-frame, and single-image tasks, with evaluations on multi-image comprehension, multi-image reasoning, video comprehension, video reasoning, single-image comprehension, and math benchmarks.

Main Findings

• Multi-image and video results (Table 1): CINEMA scores 71.6 on MUIR, 53.3 on MMIU, 36.9 on MV-MATH, 29.3 on EMMA, 55.2 on MIRB, 67.7 on Mantis, 66.5 on MVBench, 59.4 on VideoMME, and 49.0 on VideoMMMU, for an overall 54.3. Deltas versus Qwen2.5VL-7B are reported as +13.7 (MUIR), +2.7 (MMIU), +10.2 (MV-MATH), +8.9 (EMMA), +6.9 (MIRB), +3.2 (Mantis), +3.9 (MVBench), +2.7 (VideoMME), +3.2 (VideoMMMU), and +6.1 overall.

• Surpassing GPT-4o on two benchmarks: CINEMA exceeds GPT-4o on MUIR (71.6 versus 68.0) and MV-MATH (36.9 versus 32.1). The paper states it outperforms specialized video reasoning models on all three video benchmarks in the table.

• Single-image results (Table 2): The model reaches 40.6 on MMMU-Pro, 63.5 on M3COT, 25.6 on MM-IQ, 43.8 on MM-Math, 26.7 on Math-Vision, 68.7 on MathVista, 49.4 on MathVerse, and 45.5 overall, with reported deltas versus Qwen2.5VL-7B of +2.6, +3.4, +0.5, +7.4, +7.2, +3.4, +9.0, and +4.7 respectively. The text states the model surpasses GPT-4V and GPT-4o on M3COT.

• Two-stage RL raises the performance ceiling: Pass@K experiments over 7 multi-image and 7 single-image benchmarks with K ∈ {2, 4, 8, 16} show that two-stage RL consistently outperforms the baseline at pass@2, pass@4, pass@8, and pass@16.

• Two trajectories beat one, and beat conventional CoT (Table 3): Under RL, the two-trajectory cold start reaches 71.6 on MUIR, 53.3 on MMIU, and 29.3 on EMMA, versus 65.1/52.2/27.9 for a single trajectory and 70.0/51.6/26.9 for conventional CoT. Under supervised fine-tuning the two-trajectory setting also leads (58.2/51.9/24.8 versus 56.3/50.9/24.0 for single trajectory and 59.0/49.9/21.2 for conventional CoT). Directly prompting the untrained model to use the meta-actions performs poorly (33.8/36.9/14.1), which the authors present as evidence that the data construction and training are necessary.

• Every meta-action matters (Table 4): Removing any single action degrades results. Full model: 71.0 on MUIR, 55.7 on MIRB, 61.0 on VideoMME. Removing global: 63.4/52.6/57.1. Removing focus: 61.6/53.2/57.5. Removing hint: 63.5/52.3/56.8. Removing think: 60.1/53.6/57.1.

• Different image counts and unseen tasks: On MUIR (2–9 images) and MMIU (2–32 images), the model outperforms the base model in most cases and still yields a significant improvement when the input exceeds 17 images. On MUIR's 12 tasks, improvements appear on almost all tasks, including Geographic, Cartoon, and Visual Grounding, which the authors state were not in the training set.

• Entropy behavior (Figure 4): The diversity-preserving stage maintains moderate entropy, which declines during annealing; the method keeps higher entropy than the baseline while matching it on training accuracy, and produces visibly richer meta-action trajectory types after annealing.

Methodology in Plain English

The authors start from the observation that people solve multi-image problems in stages: they read the whole question, zoom in on what matters, note the traps, reason it through, then answer. They encode those stages as five explicit tags the model must emit in its response, turning an unstructured chain of thought into a constrained, checkable action sequence.

To create training data, they use a student-teacher setup. A smaller model attempts each problem and produces a trajectory, correct or not. GPT-4o then re-reasons along that same action path and produces a correct trajectory. A retrieval step searches a database of stored trajectories for a second correct path that is as dissimilar as possible from the first, so each problem has two genuinely different worked solutions. Problems GPT-4o cannot solve even with this help are set aside for reinforcement learning rather than supervised training, giving the RL stage hard examples.

Training proceeds in three phases. First, supervised cold start on the two-trajectory data for two epochs at a learning rate of 1e-5. Second, 700 RL steps using DAPO with a custom reward that rewards correct answers but subtracts a penalty proportional to how many trajectories in the sampling group share the same meta-action pattern, which pushes the model away from repeating one stock strategy. Third, 300 RL steps of the same optimizer with an annealing schedule, gradually letting the model exploit what it has learned. Rewards use math_verify and mathruler for math answers and exact string matching elsewhere, plus a format reward requiring a valid meta-action trajectory. Rollouts use batch size 64, temperature 1.0, and 8 rollouts per prompt, with an update batch size of 32. At inference, decoding uses temperature 0.6, top-p 0.7, and a maximum of 1024 generated tokens.

Why This Matters

The work argues that the structural gap between single-image and multi-image competence in MLLMs can be closed by architecture-agnostic reasoning structure plus diversity-aware RL, rather than by scaling the model. Because the backbone is a 7B open model and the reported gains include beating GPT-4o on two benchmarks, the result suggests that reasoning scaffolding and training recipe can substitute for raw scale in multi-image settings. The finding that gains transfer to tasks absent from training (Geographic, Cartoon, Visual Grounding) and to single-image and video benchmarks supports generalizability rather than benchmark-specific overfitting.

Real-world applications implied by the paper's framing:

• E-commerce: comparing product images, matching listings, or reasoning across a gallery of shots of the same item.
• Autonomous driving: integrating multiple camera views and temporal frames into one coherent scene judgment.
• Video content understanding: reasoning over sequential frames, where the reported VideoMME and VideoMMMU results are the relevant evidence.
• Educational and document workflows: multi-page or multi-figure problem solving, which the MV-MATH and EMMA results address.

Industry relevance: the authors are affiliated with ByteDance and East China Normal University, and the training pipeline is built on an open 7B backbone with a described data construction strategy, making the recipe plausible to reproduce in production settings where multi-image inputs are already routine.

Future Directions

• Scaling and backbone generality: only Qwen2.5VL 7B is used as the backbone; whether the same meta-action and two-stage RL recipe transfers to substantially larger or different model families is not established.
• Teacher dependence: cold-start data generation relies on GPT-4o producing a correct refined trajectory, which determines which problems become supervised examples and which become RL examples; sensitivity to the teacher's coverage is not reported.
• Meta-action set design: the five actions are fixed by human inspiration and validated only by removal ablations; whether additional, task-specific, or learned actions would help is left open.
• Reward and verification limits: non-math answers are scored by exact string matching, and the paper does not report how far that restricts applicability to open-ended multi-image tasks.

Target Audience

Researchers and engineers working on multimodal LLMs who are interested in multi-image and video reasoning, structured chain-of-thought decomposition, or reinforcement learning post-training for vision-language models. It is most useful to readers already comfortable with RL fine-tuning of language models, since the reward design, entropy preservation, and DAPO-based annealing are the technical core. Practitioners building multi-image production systems will find the data-construction procedure and benchmark tables actionable even without following the RL details.

Authors’ abstract

While Multimodal Large Language Models (MLLMs) excel at single-image understanding, they exhibit significantly degraded performance in multi-image reasoning scenarios. Multi-image reasoning presents fundamental challenges including complex inter-relationships between images and scattered critical information across image sets. Inspired by human cognitive processes, we propose a Cognition-Inspired Meta-Action Framework (CINEMA), which decomposes multi-image reasoning into five structured meta-actions: Global, Focus, Hint, Think, and Answer, explicitly modeling the sequential cognitive steps humans naturally employ. For cold-start training, we introduce a Retrieval-Based Tree Sampling strategy that generates high-quality meta-action trajectories to bootstrap the model with reasoning patterns. During reinforcement learning, we adopt a two-stage paradigm: an exploration phase with Diversity-Preserving Strategy to avoid entropy collapse, followed by an annealed exploitation phase with DAPO to gradually strengthen exploitation. To train our model, we construct a dataset of 56k cold-start and 58k reinforcement learning instances spanning multi-image, multi-frame, and single-image tasks. We conduct extensive evaluations on multi-image reasoning benchmarks, video understanding benchmarks, and single-image benchmarks, achieving competitive state-of-the-art performance on several key benchmarks. Our model surpasses GPT-4o on the MUIR and MVMath benchmarks and notably outperforms specialized video reasoning models on video understanding benchmarks, demonstrating the effectiveness and generalizability of our human cognition-inspired reasoning framework.