CogFlow: Bridging Perception and Reasoning through Knowledge Internalization for Visual Mathematical Problem Solving

AI summary

Overview

• Research area: Computer Vision / multimodal large language models (MLLMs), specifically visual mathematical problem solving and multimodal reinforcement learning.
• Technical level: Advanced. The paper assumes familiarity with reinforcement learning for LLMs (GRPO, PPO-style clipping, DPO variants), reward modeling, and multimodal benchmark evaluation.
• Scope: A cognitive-inspired three-stage training framework (perception → internalization → reasoning) with three new reward/optimization mechanisms and a new 120K+ annotated dataset, evaluated across six visual mathematical reasoning benchmarks.

What This Paper Is About

Multimodal large language models still fail on visual math problems, and recent work identifies weak visual perception as the bottleneck. However, existing fixes only try to extract visual information better — they never check whether that extracted information is actually used faithfully in the following reasoning steps. The authors name this failure "reasoning drift" (chains that look coherent but contradict the diagram) and build CogFlow to close the gap by inserting an explicit knowledge-internalization stage between perception and reasoning.

Key Contributions

1. CogFlow framework: A three-stage, cognitively inspired pipeline modeled on the human flow of perception ⇒ internalization ⇒ reasoning, where the internalization stage is made explicit rather than left implicit in a reasoning chain.
2. Synergistic Visual Rewards (SynVRs): A combination of a Visual Parameterized Reward (VPR) that matches geometric primitives (points, lines, circles) in a parameter space via Hungarian matching, and a Visual Semantic Reward (VSR) that compares re-rendered images using a frozen FG-CLIP encoder and cosine similarity — giving both local geometric fidelity and global layout/style coherence.
3. Knowledge Internalization Reward (IntlzR): A trained reward model that detects reasoning drift. It is trained on a positive–negative dataset where each positive is paired with five negatives synthesized from five typical internalization failures (omitting/misbinding primitives, introducing nonexistent facts, misusing external theorems, contradicting geometric constraints, referring inconsistently to elements), optimized with Softmax-DPO.
4. Visual-Gated Policy Optimization (VGPO) and the MathCog dataset: VGPO adds a visual gate that filters perception trajectories by a perceptual-accuracy score before reasoning generation, combined with an outcome-supervised Inference Reward. MathCog supports training with three subsets containing over 120K high-quality perception-reasoning aligned annotations.

Main Findings

• Headline accuracy: CogFlow-7B reaches 66.0% accuracy on FlowVerse, 53.9% on MathVerse (testmini), 76.8% on MathVista, 64.1% on WeMath, 58.1% on LogicVista, and 46.2% on DynaMath.
• Gains are largest where vision dominates: On FlowVerse, Vision Dense accuracy is 42.7% and Vision Centric is 61.1% (the prose also cites 55.6% for Vision Primary, while Table 1 lists 63.5% for that column). On MathVerse the prose cites 44.8% (Vision Intensive), 42.1% (Vision Dominant) and 25.7% (Vision Only), while Table 2 lists 42.4%, 44.8% and 26.3% respectively — the subset figures in the text and tables do not fully line up.
• Comparison to closed-source models: The paper states CogFlow achieves on-par or better results than much larger closed-source MLLMs in several settings, but the tables show it below several of them — GPT-5 reaches 68.2% CoT-E / 59.3% accuracy on FlowVerse and 71.1% on WeMath; Gemini-2.5-Pro reaches 78.0% on WeMath, 73.8% on LogicVista and 56.3% on DynaMath; Doubao-pro-1.5 reaches 79.5% on MathVista versus CogFlow's 76.8%. The text also claims LogicVista performance "trailing only GLM-4.IV-9B," though Table 4 lists GLM-4.5V at 62.4%, Doubao-1.5-Pro at 64.2%, GPT-5 at 70.0% and Gemini-2.5-Pro at 73.8%.
• Every component helps (Table 5, FlowVerse / MathVerse): Baseline SFT+GRPO scores 57.4 / 48.2 CoT-E and 48.7 / 35.6 accuracy. Adding SynVRs alone → 63.2 / 50.5 and 54.7 / 36.9; IntlzR alone → 62.7 / 49.9 and 53.5 / 36.2; VGPO alone → 63.4 / 50.8 and 54.8 / 37.3; SynVRs+IntlzR → 64.4 / 52.1 and 55.1 / 38.0; all three → 66.0 / 53.9 and 56.2 / 39.5. VGPO is described as the single most influential module.
• VPR and VSR are complementary: Combining both improves CoT-E by up to +3.0% and accuracy by +1.7% on MathVerse, and CoT-E by +2.2% and accuracy by +2.1% on FlowVerse over the no-visual-reward baseline.
• All five error types matter for IntlzR: Removing any single error type degrades performance; the largest drops come from excluding omission/misbinding of primitives or contradiction of geometric constraints. Softmax-DPO reaches 66.0% / 56.2% on FlowVerse, outperforming vanilla DPO.
• The visual gate helps at inference alone: Using the gate only at inference yields roughly 0.6–1% absolute accuracy gain even without VGPO training; VGPO adds a larger improvement by shaping the policy during training.
• Error-type analysis (classified with GPT-5 on FlowVerse): Adding SynVRs reduces Perception Error by −2%; adding IntlzR reduces Knowledge-Internalization Error by −2% relative to the baseline. GPT-4o's strength is attributed mainly to reasoning, with diagram perception lagging behind specialized 7B models, and neither MathFlow-7B nor VLM-R1 meaningfully reduces Knowledge Internalization Error.

Methodology in Plain English

The model is first fine-tuned with supervised learning on the MathCog-SFT subset to acquire basic perception and reasoning skills, then optimized with reinforcement learning on MathCog-RL.

During RL, for each question the model samples several candidate trajectories. Each trajectory is split into a perception part (a structured parse of diagram primitives and relations) and a reasoning part conditioned on that parse.

Three signals score the trajectories:

1. SynVRs check whether the perception is right. VPR turns predicted shapes into parametric equations (for example, a circle primitive is rewritten as an algebraic equation) and compares them against ground-truth primitives using optimal one-to-one matching. VSR re-renders the predicted text into an image and compares it to the ground-truth image with a frozen FG-CLIP encoder using cosine similarity. A weight α (between 0 and 1) balances the two.
2. IntlzR checks whether the reasoning faithfully builds on what was perceived. A separate 3B reward model, trained on positive trajectories paired with five types of synthetic negatives using Softmax-DPO, scores how well the chain stays anchored to the internalized representation.
3. InfR checks the final answer's correctness and output format.

A visual gate scores each perception attempt and accepts the first one above a threshold τ, regenerating candidates up to M times if needed; if none pass, the highest-scoring attempt is used. These rewards are combined and optimized with a PPO-style objective using group-normalized advantages and a KL penalty to a frozen reference policy.

Training used Qwen2.5-VL-7B as the base model and Qwen2.5-VL-3B for the IntlzR reward model: SFT for 2 epochs at learning rate 1×10⁻⁵ and batch size 64, RL for 1 epoch at 1×10⁻⁶ and batch size 16, and the reward model for 3 epochs at 7×10⁻⁶ and batch size 64, all on 16 NVIDIA A100 GPUs.

Why This Matters

• Research impact: It reframes the visual-math bottleneck from "can the model see?" to "does the model use what it saw?", and demonstrates that reward modeling can target an intermediate cognitive stage rather than only final answers or perception quality.
• Real-world applications (potential, not reported as tested in the paper):
  • Automated tutoring or homework assistants that must explain a geometry solution step by step from a diagram.
  • Automated grading of diagram-based math work, where the system must check whether a student's steps are consistent with the figure.
  • Accessibility tools that turn a diagram into an accurate structured verbal description for learners who cannot see it.
  • Diagram understanding in technical documents such as engineering figures and plotted algebraic curves.
• Industry relevance: The approach targets small open 7B models, which matters for cost-sensitive deployment where closed-source models such as GPT-5, Gemini-2.5-Pro or Doubao-pro-1.5 are impractical, and it provides a decoupled perception/reasoning interface that is easier to audit and debug than a single unstructured chain.

Future Directions

• Reducing dependence on ground-truth geometry: VPR relies on ground-truth primitives and Hungarian matching during training, which limits it to settings where structured annotations exist. Extending it to unannotated diagrams is an open problem.
• Generalizing beyond visual math: The internalization reward is built from five math-specific failure modes; whether analogous reward models transfer to physics, chemistry, chart or table reasoning is untested.
• Closing the gap to large closed-source models: CogFlow-7B trails Gemini-2.5-Pro and GPT-5 on several benchmarks (for example 64.1% versus 78.0% and 71.1% on WeMath), so scaling the base model or the reward model is a natural next step.
• Resolving reported inconsistencies: The subset-level numbers in the prose differ from the corresponding table entries on FlowVerse and MathVerse, and the LogicVista claim in the text conflicts with Table 4, so independent reproduction would clarify the true profile of gains. The paper does not report inference cost or latency for the multi-candidate visual gate procedure.

Target Audience

Researchers and engineers working on multimodal LLMs, reinforcement learning for reasoning, or mathematical/geometric diagram understanding. It will also interest practitioners building educational technology or document-understanding systems who need grounded, auditable reasoning chains rather than unconstrained chain-of-thought output. Readers without a background in RL-based post-training will need to consult the cited GRPO, DPO and Softmax-DPO references first.

Authors’ abstract

Despite significant progress, multimodal large language models continue to struggle with visual mathematical problem solving. Some recent works recognize that visual perception is a bottleneck in visual mathematical reasoning, but their solutions are limited to improving the extraction and interpretation of visual inputs. Notably, they all ignore the key issue of whether the extracted visual cues are faithfully integrated and properly utilized in subsequent reasoning. Motivated by this, we present CogFlow, a novel cognitive-inspired three-stage framework that incorporates a knowledge internalization stage, explicitly simulating the hierarchical flow of human reasoning: perception$\Rightarrow$internalization$\Rightarrow$reasoning. In line with this hierarchical flow, we holistically enhance all its stages. We devise Synergistic Visual Rewards to boost perception capabilities in parametric and semantic spaces, jointly improving visual information extraction from symbols and diagrams. To guarantee faithful integration of extracted visual cues into subsequent reasoning, we introduce a Knowledge Internalization Reward model in the internalization stage, bridging perception and reasoning. Moreover, we design a Visual-Gated Policy Optimization algorithm to further enforce the reasoning is grounded with the visual knowledge, preventing models seeking shortcuts that appear coherent but are visually ungrounded reasoning chains. Moreover, we contribute a new dataset MathCog for model training, which contains samples with over 120K high-quality perception-reasoning aligned annotations. Comprehensive experiments and analysis on commonly used visual mathematical reasoning benchmarks validate the superiority of the proposed CogFlow. Project page: https://shchen233.github.io/cogflow.