When to Rethink: Learning Multi-Perspective Self-Verification for Vision-Language Models

AI summary

Overview

Research area: Multimodal reasoning and self-verification for vision-language models (VLMs), combining reinforcement-learning-based answer generation with learned reliability estimation.

Technical level: Advanced — the method assumes familiarity with GRPO-style policy optimization, binary cross-entropy verification losses, and selective inference with thresholds.

Scope: The paper diagnoses why existing self-verification for VLMs is unreliable (verifier strength and prompt sensitivity) and proposes MOTIVE, a multi-view self-verification framework with a learned reliability score that decides when to accept an answer and when to rethink.

What This Paper Is About

Vision-language models often produce answers that look plausible but are wrong, and asking the same model to check its own answer is a cheap way to catch these errors — except current self-verification methods depend on one fixed criterion or prompt, so their reliability estimates are incomplete and unstable. The authors first measure how verifier capability and prompt wording change verification outcomes, then build a method that checks each candidate answer from several complementary verification perspectives and learns a correctness-aligned reliability score. That score decides, at inference time, whether to return the answer or spend another reasoning turn rethinking it, without calling an external judge model.

Key Contributions

1. A systematic preliminary analysis of self-verification. The authors compare direct answering against verification with InternVL3 judges of 2B, 8B, 14B, and 38B parameters on VQA, ScienceQA, and TableVQA, and separately compare five Direct Verification Prompts (DVP-1–DVP-5) against generated verification questions (GVQ-1–GVQ-3) on Qwen2.5-VL-3B.

2. MOTIVE, a multi-view self-verification framework. Each candidate answer is scored by K = 5 complementary verification views whose YES-probabilities are averaged into a single reliability score, supported by a proposition showing that averaging reduces variance when view errors are not perfectly correlated.

3. Correctness-grounded verification learning. View-level, aggregate-level, and calibration losses are supervised by automatically derived candidate-correctness labels, combined with a GRPO objective and a computation-aware reward that penalizes unnecessary rethinking turns.

4. Reliability-guided selective rethinking and broad evaluation. A threshold on the learned score governs an accept-or-rethink decision with history-guided revision; experiments cover five benchmarks (ReFocus, VQA, SLaKE, ScienceQA, TableVQA) and three backbones (Qwen2.5-VL-3B, Qwen2.5-VL-7B, InternVL3-2B).

Main Findings

• Verification helps, and stronger verifiers help more: adding an external Yes/No judge consistently improves final-answer accuracy over direct answering for both Qwen2.5-VL backbones on all three preliminary benchmarks, and larger InternVL3 judges generally yield better final performance than smaller ones.

• Self-verification is highly prompt-sensitive: with Qwen2.5-VL-3B, changing only the verification prompt shifts accuracy noticeably — for example VQA ranges from 47.23 (DVP-1) to 52.36 (DVP-3), TableVQA from 53.66 (GVQ-1) to 57.00 (DVP-4 and DVP-5), and ScienceQA from 80.58 (GVQ-3) to 82.20 (DVP-3).

• No single direct prompt dominates: the best DVP varies by benchmark (DVP-3 on VQA and ScienceQA, DVP-4/DVP-5 on TableVQA), which the authors use to justify combining complementary views rather than picking one prompt.

• Generated verification questions are unstable: GVQ-style prompting is generally less stable than direct verification and degrades as more auxiliary questions are generated (VQA 51.72 → 50.64 → 50.21 for GVQ-1 to GVQ-3).

• MOTIVE leads on all five benchmarks and all three backbones: average accuracy is 63.13 for Qwen2.5-VL-3B (baseline 55.11, best baseline ADPO 60.41), 68.63 for Qwen2.5-VL-7B (baseline 60.49, ADPO 66.01), and 65.18 for InternVL3-2B (baseline 56.50, ADPO 62.17).

• Better accept-or-rethink decisions: Self-Verification Accuracy rises from 68.91 to 77.48 (Qwen2.5-VL-3B), 72.32 to 79.69 (Qwen2.5-VL-7B), and 70.07 to 78.50 (InternVL3-2B) relative to conventional Self-Verification.

• Fewer reasoning turns: under a maximum budget of T = 3, average turns drop from 2.58 to 2.29 (Qwen2.5-VL-3B), 2.46 to 2.23 (Qwen2.5-VL-7B), and 2.53 to 2.26 (InternVL3-2B); reductions are larger on VQA and SLaKE and smaller on ScienceQA.

• View diversity beats redundancy: replacing repeated copies of DVP-1 with distinct prompts improves accuracy from one distinct view up to the full set of five, with gains gradually saturating.

• All three loss components contribute: on Qwen2.5-VL-3B, adding the view loss lifts the GRPO-only average from 55.11 to 60.60, adding the aggregate loss reaches 62.42, and adding the calibration loss reaches 63.13; the same ordering holds for the other two backbones.

• Continuous averaging beats majority vote: averaging YES probabilities outperforms majority voting on every backbone (for example 63.13 vs 62.68 on Qwen2.5-VL-3B, 68.63 vs 68.22 on Qwen2.5-VL-7B, 65.18 vs 64.78 on InternVL3-2B).

• Loss weights matter: the best settings are λ_ver = 0.1, β_agg = 1.0, and β_cal = 0.1; removing the verification objective, the aggregate-level loss, or the calibration loss each reduces accuracy, while larger weights slightly degrade it.

Methodology in Plain English

The authors start by testing two hypotheses. First, they let a backbone model answer a question, then hand the image, question, and answer to a separate judge model that must say Yes or No; a No sends the model back to try again. Second, they hold the model and data fixed and swap only the checking instruction, comparing five hand-written "direct verification" prompts with variants that first generate one, two, or three auxiliary verification questions.

Those results motivate the main method. Each candidate answer is scored by five different verification prompts, each producing a probability that the answer is correct. The five numbers are averaged into one reliability score. Because the training data gives ground-truth answers, the system can automatically label each intermediate candidate as correct or incorrect and train the verifier directly on those labels: one loss trains each view separately, one trains the averaged score, and a squared-error term keeps the averaged score on a sensible numerical scale for thresholding.

Training also uses GRPO, a reinforcement-learning procedure that rewards correct final answers while subtracting a penalty for each extra rethinking turn, so the model learns to stop early when it can. The answer-generation and verification losses are applied to different tokens, keeping answer quality and reliability estimation separate.

At inference, if the averaged reliability score reaches 0.5 the answer is returned. Otherwise the previous answer, the five verification verdicts, and a "rethink" instruction are added to the conversation history, and the model produces a revised answer — up to three turns, after which the last candidate is returned. Training used AdamW at a learning rate of 1e-6 in BF16, rollout group size 8, sampling temperature 1.0, and verification loss weight 0.1.

Why This Matters

Reliability estimation that stays inside a single model removes the cost and complexity of running a separate judge, which matters for deployment where an extra model is impractical. The paper also reframes self-verification as a learned, calibrated signal rather than a prompt trick, and shows that the same machinery can cut the number of reasoning turns — a direct efficiency benefit, not just an accuracy one.

Real-world applications:

• Medical visual question answering: SLaKE is a radiology VQA benchmark, and knowing when a model's answer should be trusted is central to clinical decision support.
• Document and table understanding: ReFocus and TableVQA target charts, tables, and structured images, relevant to financial reporting and enterprise document workflows.
• Scientific and educational tutoring: ScienceQA covers image-grounded science questions, where plausible-but-wrong explanations are a real risk for learners.
• General visual assistants: any multimodal assistant that must decide between answering immediately and spending more compute on a hard case.

Industry relevance: the accept-or-rethink mechanism is a compute-allocation policy, so it maps directly onto serving systems that want to trade latency against accuracy per query, and onto on-device or cost-sensitive deployments where an external judge model is not affordable.

Future Directions

• Removing the manual prompt set. The five verification views are predefined prompts; learning or generating diverse views automatically, and testing whether more than five views keeps helping given the saturating diversity curve, is an open question.

• Threshold and budget policy. The acceptance threshold is fixed at τ = 0.5 and the budget at T = 3; adapting either per query or per dataset is unexplored.

• Beyond accuracy: calibration quality. The paper measures accuracy, self-verification accuracy, and turn counts; broader calibration diagnostics for the reliability score are not reported.

• Generalization of the learned verifier. Results cover three backbones and five benchmarks; whether the correctness-grounded verification signal transfers to unseen task types, or to video and other modalities, is not established.

Target Audience

Researchers and practitioners working on multimodal reasoning, self-correction, and inference-time compute allocation for VLMs; reinforcement-learning engineers interested in combining GRPO with auxiliary verification objectives; and applied teams building visual assistants that need a cheap, in-model signal for when to trust an answer versus retry. Readers without background in policy-gradient training will find the preliminary analysis in Section 2 accessible on its own, but the method sections assume familiarity with reward-based fine-tuning.

Authors’ abstract

Vision-language models (VLMs) have achieved strong performance in multimodal reasoning, yet they remain prone to generating plausible but incorrect answers. Self-verification offers a practical way to improve answer reliability without relying on external judges, but existing methods typically depend on a single verification criterion or fixed prompt, resulting in incomplete and unstable reliability estimates. We first systematically analyze how verifier capability and prompt design affect verification performance. Our findings show that stronger verifiers provide more reliable judgments, while verification performance is highly sensitive to prompt choice, with no single prompt consistently dominating across tasks. Guided by these findings, we propose \texttt{MOTIVE}, a \textbf{M}ulti-View Self-Verificati\textbf{O}n wi\textbf{T}h Rel\textbf{I}ability-Guided Selecti\textbf{VE} Rethinking framework for reliable multimodal reasoning. \texttt{MOTIVE} evaluates each candidate answer from complementary verification perspectives and learns a correctness-aligned reliability score through correctness-grounded multi-view verification learning. During inference, this score governs an accept-or-rethink decision, allowing reliable answers to be returned directly while uncertain ones trigger history-guided rethinking. Extensive experiments across diverse multimodal benchmarks and VLM backbones demonstrate that \texttt{MOTIVE} consistently outperforms strong self-verification and self-correction baselines. Further results show that reliable verification improves accept-or-rethink decisions and reduces unnecessary reasoning turns, enabling more reliable and efficient self-verification without an external judge.