Learning Where Outcomes Change:Credit-Addressable Reasoning for Multimodal Geometry

AI summary

Overview

Research area: Multimodal machine learning — vision-language models (VLMs) for geometry reasoning, combining structured chain-of-thought representation design with fine-grained reinforcement learning (credit assignment).

Technical level: Advanced. The paper assumes familiarity with reinforcement learning for language models (GRPO-style policy optimization, advantages, rollouts), supervised fine-tuning, and multimodal benchmark evaluation.

Scope: The paper proposes "credit-addressable reasoning," a principle in which the semantic units a model exposes during inference also define the units at which learning compares alternatives and assigns credit, instantiated as Code-CoT (line-addressable executable code plus typed reasoning events) and CE-GRPO (event-level group-relative policy optimization).

What This Paper Is About

Geometry reasoning with vision-language models is fragile: a single mistake in object binding, angle interpretation, or auxiliary construction can invalidate every later deduction. In standard free-form reasoning traces these decisive choices are invisible, and trajectory-level reinforcement learning spreads one terminal reward across the whole response, so the model cannot tell which intermediate decision caused success or failure. The paper's goal is to make those decisions explicit, addressable units that are used both for reasoning and for localized credit assignment.

Key Contributions

1. The credit-addressable reasoning formulation. The authors define a principle in which the semantic units exposed during inference also define where optimization compares alternatives and assigns credit, motivated by a controlled study of six models showing why code is a suitable shared representation.

2. Code-CoT. A reasoning space that retains the original diagram, converts it to line-addressable executable Matplotlib perception code, and organizes the rest of the solution into typed events — think, reference, auxiliary, and coordinate — each of which is both a checkable geometric operation and an addressable credit unit.

3. CE-GRPO (Critical-Event Group Relative Policy Optimization). A method that selects candidate events using a structural prior plus type-normalized entropy, fixes the image, question and complete prefix, samples multiple complete continuations through the final answer, and converts terminal outcome differences into event-conditioned advantages — without process annotations or an auxiliary value model.

4. Empirical validation across nine geometry benchmarks, including per-benchmark gains, closed-only evaluation, selector ablations, diagram-to-code fidelity analysis, and event-count analysis supporting the mechanism.

Main Findings

• Strong aggregate accuracy. Across nine geometry benchmarks, CE-GRPO reaches an average accuracy of 76.04, outperforming the native Qwen3-VL-8B backbone, Code-CoT SFT, and trajectory-level GRPO by 8.09, 6.49, and 3.43 points respectively, and improving over the backbone on all nine benchmarks. It also exceeds the strongest fine-grained baseline by 6.73 points.

• The Code-CoT protocol must be learned, not prompted. Code-CoT prompting alone yields 49.26 average accuracy, 18.69 points below the native backbone, whereas supervised fine-tuning on the protocol raises the average to 69.55.

• Code complements rather than replaces the diagram. In the six-model controlled study (Table 1), combining diagram and externally generated code (I+C) was best for all six models. For example: InternVL3.5-8B 45.5 → 63.9 (+18.4), Qwen2.5-VL-7B 49.4 → 60.6 (+11.2), Qwen3.5-9B 79.9 → 85.4 (+5.5), Kimi-K2.6 88.2 → 92.6 (+4.4), Claude-Opus-4.8 87.9 → 91.1 (+3.2), Gemini-3.1-Pro 92.3 → 93.0 (+0.7). Combining both modalities reduced the Qwen2.5-VL-7B Text-Dominant–Vision-Only gap from 13.3 to 3.0 points.

• Reliable code generation is the bottleneck. Under the same code-only setting, self-generated code trails externally generated code by 11.7–16.6 points for every open model.

• Gains concentrate in dependency-heavy tasks. Relative to trajectory-level GRPO, CE-GRPO gains 15.16 points on GeoLaux-mini and 9.44 on MM-Math, where intermediate constructions and decisions affect multiple later steps, while gains are smaller or mixed on visual-grounding and standard plane-geometry tasks.

• A single-call model beats two-stage systems. CE-GRPO surpasses GDP-4B-RL → Qwen3-VL-8B by 3.07 points on average, winning seven of nine benchmarks, and outperforms GeoTikzBridge-8B → Qwen3-VL-8B on all nine. GDP leads only on Geometry3K and PGPS9K.

• Improvements are in solution quality, not just protocol compliance. On validly terminated responses, CE-GRPO holds a 3.91-point mean gain over trajectory-level GRPO (mean closed-only accuracy 75.92 → 79.82), outperforming on seven of nine benchmarks, with the largest gains on GeoLaux-mini (+12.53), MM-Math (+8.42), and PGPS9K (+5.07).

• The modality gap roughly halves. On the 788 MathVerse problems spanning text-dominant to vision-only variants, CE-GRPO leaves Text-Dominant nearly unchanged (−0.88) but gains 15.10 points on Vision-Only, reducing the TD–VO gap from 30.07 to 14.09 and the standard deviation from 9.91 to 4.87.

• Structure and entropy both matter, and structure matters more. Entropy alone gives only a 0.35-point gain over random selection. Structure alone raises the average to 74.26 and cuts the unclosed-response rate from 12.31% to 7.07%. Combining both reaches 76.04 with the lowest unclosed rate of 4.73%, with pronounced gains over structure alone on Geometry3K (+5.43) and GeoLaux-mini (+3.34).

• Training improves the executable visual representation itself. On 100 MathVerse-TD problems, macro recall of diagram-to-code fidelity rises from 55.21% (base + prompt) to 70.16% (Code-CoT SFT) to 80.43% (CE-GRPO); render success rises from 89.0% to 95.0% to 99.0%, and recall on successful renders from 62.0% to 73.9% to 81.2%.

• Advantage widens with reasoning length. CE-GRPO's margin over trajectory-level GRPO increases by 3.77 points per additional intermediate event (r = 0.866, exact p = 0.0016), remaining significant under leave-one-benchmark-out tests (p ≤ 0.028). The widening is driven by trajectory-level GRPO, whose gain over SFT declines by −5.55 points per event (p = 0.0062), whereas the CE-GRPO trend is weaker and not significant (−1.78 points/event, p = 0.2048).

• Offline selector analysis. Structural selection identifies outcome-changing events about 30% more often than random selection.

Methodology in Plain English

Step 1 — Establish that code is a useful reasoning space. The authors take MathVerse, which presents each problem in five variants shifting information from text to the diagram, and test three input conditions on three open and three proprietary models: diagram only, externally generated code only (with the diagram withheld), and both. They run a fourth condition for open models: self-generated code only, with the diagram withheld, so it differs from external code only in who wrote the code. The result — both modalities together win for all six models, and self-generated code lags external code — tells them code should sit alongside the image and must be learned rather than prompted.

Step 2 — Build the reasoning space (Code-CoT). The model generates a line-numbered, executable Matplotlib program describing the diagram inside a <perception> block, then a one-line plan, then an alternating sequence of think events and typed actions (reference retrieves supporting code lines, auxiliary extends the working geometry, coordinate sets up an executable coordinate frame), ending in an <answer> block. The image is kept throughout. Because the protocol tags are deterministic, a simple parser can split the trace into complete, typed events and exactly recover the prefix preceding any event — no learned segmenter or step detector needed. Programmatic checkers validate structure, code executability, line grounding, and type-specific action validity.

Step 3 — Supervised fine-tuning. Traces for 18,302 curated examples are synthesized (Gemini-3.1-Pro transcribes diagrams to code; DeepSeek-V4-Pro generates plans, events, and answers), and only traces passing structural, execution, grounding, action-validity, and answer-correctness checks are retained. SFT starts from Qwen3-VL-8B-Instruct with the visual encoder frozen.

Step 4 — Localized reinforcement learning (CE-GRPO). A programmatic reward scores each response (correctness plus a λ = 0.3-weighted action-validity rate minus repetition and answer-leakage penalties, clipped to [−1, 1.3]; structurally invalid responses get −1). Standard GRPO computes one advantage per complete response and applies it to every token. CE-GRPO instead picks candidate event boundaries using a structural prior together with event entropy normalized within each event type (since entropy scales differ across types). For each candidate it keeps the image, question, and full prefix fixed, samples several complete continuations through the final answer, rewards each, and computes an advantage across those siblings. Because the shared prefix is treated as prompt and excluded from the policy loss, the update lands only on the regenerated event and its downstream consequences. If all continuations get the same reward, that group contributes no update — so a useless selection wastes computation rather than injecting false supervision. Ordinary and shared-prefix prompts are mixed at a 1:1 ratio under the same GRPO objective and reward.

Step 5 — Evaluation. Nine benchmarks covering visual grounding (MathVerse, VisOnlyQA-Syn, VisOnlyQA-Real, MathVista-GPS), plane geometry (Geometry3K, PGPS9K, GeoQA), auxiliary construction (GeoLaux-mini), and process-level reasoning (MM-Math), plus ablations of the selector, closed-only accuracy analysis, fidelity scoring on 100 MathVerse-TD problems with a condition-blind judge, and a regression of per-benchmark gains against event count.

Why This Matters

Impact on research. The paper argues that representation design and optimization design should be co-designed rather than treated separately. Prior program-aided methods expose reasoning structure without making it an optimization unit; prior fine-grained RL methods refine update locations but derive semantic decisions from learned values, fixed boundaries, or token statistics. CE-GRPO derives branch points from the model's own typed protocol, requires no process annotations, no learned reward model, and no auxiliary value model, and its reward is a verifier rather than a preference model. The finding that the advantage grows with reasoning length (3.77 points per additional event) directly targets the long-horizon credit-diffusion problem.

Real-world applications (as motivated and evidenced by the paper's task settings):

• Automated tutoring and homework help for geometry, where the model must read a diagram and produce an auditable, executable solution rather than an opaque answer.
• Diagram understanding in technical documents, where converting figures into executable, line-addressable descriptions provides a checkable intermediate representation.
• Engineering and CAD-adjacent workflows that require explicit auxiliary constructions and coordinate frames to be verifiable before downstream computation.
• Any multimodal reasoning pipeline with explicit intermediate structure and delayed outcome supervision, which the authors state the framework is broadly applicable to.

Industry relevance. The method builds on Qwen3-VL-8B-Instruct with a single model call and beats two-stage perception–reasoning systems (GDP-4B-RL → Qwen3-VL-8B, GeoTikzBridge-8B → Qwen3-VL-8B) that require an extra call, which is relevant where inference cost and latency matter. The programmatic reward avoids training a separate reward model.

Future Directions

• Reduce the reliance on synthesized supervision. SFT traces were generated with Gemini-3.1-Pro and DeepSeek-V4-Pro; whether the same protocol can be bootstrapped from weaker or open generators, or from reinforcement learning alone, is not reported.
• Disentangle perception errors from code-translation errors. The authors explicitly note that the self-generated versus external code comparison isolates the effect of code source but "does not separate perception errors from errors in translating a correct perception into code."
• Extend beyond geometry. The paper states the framework applies to reasoning tasks with explicit intermediate structure and delayed outcome supervision; applying it to domains without a natural typed-event protocol remains open.
• Address the residual failure modes. The unclosed-response rate for CE-GRPO (4.73% in the selector ablation, 4.72% in the closed-only table) is not zero, and CE-GRPO loses to GDP-4B-RL on Geometry3K (−4.42) and PGPS9K (−7.30), suggesting fixed symbolic parsing still favors relation-centric geometry; understanding when to combine symbolic parsing with integrated perception is an open question.

Target Audience

Researchers and graduate students working on multimodal reasoning, reinforcement learning for language models, and fine-grained credit assignment; engineers building reliable diagram-reasoning or tutoring systems who need auditable intermediate reasoning; and anyone interested in how representation design and optimization design can be aligned rather than treated as separate problems. Readers should be comfortable with GRPO-style policy optimization, group-relative advantages, and standard VLM benchmark evaluation to get full value from the results tables.

Authors’ abstract

Multimodal geometry reasoning requires VLMs to extract precise visual relations and preserve them through multi-step deduction. Existing free-form traces obscure the decisions that determine the answer, and trajectory-level reinforcement learning distributes a single terminal signal across the entire response. We introduce credit-addressable reasoning, in which the semantic units exposed during inference also define where learning compares alternatives and assigns credit. We instantiate this principle with Code-CoT, which retains the diagram, represents visual relations as line-addressable executable code, and organizes reasoning into typed events, and CE-GRPO, which selects event boundaries using structural priors and type-normalized entropy, samples complete continuations from shared prefixes, and converts outcome differences into localized advantages. Across nine geometry benchmarks, CE-GRPO achieves an average accuracy of 76.04, outperforming Qwen3-VL-8B and trajectory-level GRPO by $8.09$ and 3.43 points, respectively. Its relative advantage increases with the number of intermediate events, demonstrating the value of representation--optimization co-design for long, dependency-heavy multimodal reasoning.