Concise Geometric Description as a Bridge: Unleashing the Potential of LLM for Plane Geometry Problem Solving

AI summary

Overview

Research area: Artificial Intelligence — multimodal reasoning, specifically Plane Geometry Problem Solving (PGPS) with large language models and multimodal LLMs.

Technical level: Advanced. The paper assumes familiarity with supervised fine-tuning, Chain-of-Thought prompting, reinforcement learning with Group Relative Policy Optimization (GRPO), and formal geometric representation languages.

Scope: The paper proposes a two-stage training pipeline that teaches a Multimodal LLM (MLLM) to translate geometry diagrams into a concise formal description language (CDL), which an off-the-shelf frozen LLM then uses to solve geometry problems.

What This Paper Is About

Plane Geometry Problem Solving requires reading a diagram together with its text description and producing a correct answer. LLMs reason well but cannot see diagrams, while MLLMs can see diagrams but tend to make visual perception errors, and fine-tuning them end-to-end on large geometry datasets may erode the base LLM's reasoning ability. The paper's goal is to separate perception from reasoning: a trained MLLM converts the diagram into a concise textual geometric description, and an unmodified, frozen LLM performs all of the reasoning, using only 5.5k training samples.

Key Contributions

1. A new PGPS paradigm. The paper observes that an LLM is potentially a powerful PGPS solver when visual information is appropriately converted into text (Table 1), and therefore trains an MLLM Interpreter to produce geometric descriptions (CDL) while an off-the-shelf LLM Solver does the reasoning.
2. A two-stage MLLM Interpreter training pipeline. CoT-augmented Supervised Fine-Tuning followed by GRPO, using newly designed CDL matching rewards instead of a conventional solution-based reward. The paper reports two technical insights: concise geometric descriptions ease interpreter learning by narrowing the search space, and CDL matching rewards give denser, more direct guidance than solution-based rewards.
3. A new dataset, Formalgeo7k-Rec-CoT. Built by rigorous manual review of Formalgeo7k v2 to fix annotation errors and by parsing CDL annotations to generate Chain-of-Thought sequences. The review was performed by four qualified annotators, with each item independently reviewed by two annotators, consensus required for a modification, and a third annotator adjudicating discrepancies.
4. Strong results with limited data. Trained on only 5.5k data, the method outperforms all compared open-source MLLMs and reaches performance comparable to leading closed-source MLLMs on Formalgeo7k-Rec-CoT, Unigeo, and the geometric set of MathVista.

Main Findings

• LLMs are strong geometry solvers when given good text. In Table 1, Claude-Opus-4.1 scores 69.1 with visual input, 76.8 with generated captions, and 84.2 with ground-truth CDL; Gemini2.5-Pro scores 81.8, 81.1, and 84.3; and the LLM Qwen3 30B scores 82.1 with captions and 88.4 with ground-truth CDL.
• CDL generation quality improves substantially with RL. On Formalgeo7k-Rec-CoT, Qwen2.5-VL 7B goes from ConsCDL recall/precision of 70.0/68.3 after SFT to 92.7/92.1 after RL, and Qwen3-VL 8B reaches 95.9/95.8 ConsCDL recall/precision after RL.
• The unified interpreter beats specialized parsers. Fgeo-Parser (Diag.) trained on 14.7k samples reaches ImgCDL recall 77.5 and ConsCDL recall 87.0; Diagram Formalizer trained on 1M reaches ImgCDL recall 92.9 and ConsCDL recall 90.3; the proposed interpreter trained on 5.5k reaches ImgCDL recall 96.9 and ConsCDL recall 92.7 (Qwen2.5-VL 7B, RL).
• Solving accuracy beats open-source MLLMs and rivals closed-source ones. With 5.5k data the method scores 85.7 on Formalgeo, 84.0 on Unigeo, and 80.8 on MathVista, versus Gemini2.5-Pro at 81.8, 84.6, and 81.3, and Qwen2.5-VL 32B at 57.3, 67.9, and 66.8. On Formalgeo-Rec-CoT the method surpasses Gemini2.5-Pro by 3.9% accuracy.
• Concise descriptions generalize better than expanded ones. Introducing redundant ConsCDL terms lowers ConsCDL recall/precision from 92.7/92.1 to 91.6/90.7 and solving accuracy from 83.2 to 80.5. The expanded version scores slightly higher on the training set but generalizes worse on the validation set, which the authors attribute to memorization rather than learning generalizable patterns.
• CDL matching rewards beat solution-based rewards. Adding a solution-based reward from the LLM Solver (Qwen3 30B) on top of the CDL matching rewards drops accuracy from 83.2 to 81.3; with the solution-based reward alone, the paper reports no performance gain compared with SFT.
• Every designed reward contributes. Using only the ConsCDL reward gives 81.0 accuracy; adding ImgCDL raises it to 82.2; adding TextCDL gives 81.6; using the full set (format, ConsCDL, ImgCDL, TextCDL rewards) gives 83.2.
• CoT should be limited to ConsCDL. Training without CoT yields only 76.3 accuracy; adding ConsCDL CoT raises it to 83.2; adding ImgCDL or TextCDL CoT instead lowers it to 82.8 and 81.7 respectively.
• Rollout number matters for cost and performance. In GRPO, N=8 gives 83.2 accuracy versus 80.9 at N=5 and 82.5 at N=10, and N=10 costs an extra 80 hours of training time versus N=8.
• The approach generalizes across LLM Solvers. With CDL generated by the interpreter (Qwen2.5-VL 7B), accuracy on Formalgeo-Rec-CoT is 83.2 with Qwen3 30B, 82.5 with Qwen3 32B, 82.3 with GLM-4.6, and 79.4 with DeepSeek V3.1.
• Gains hold against the same solver. Using the same LLM solver (Qwen3 30B), DFE-GPS's Diagram Formalizer scores 80.3 on Formalgeo, 79.9 on Unigeo, and 69.0 on MathVista, while the proposed method scores 85.7, 84.0, and 80.8.
• Improvements are not solely from cleaned data. Training and evaluating on the original Formalgeo7k v2 still yields TextCDL 98.2/98.3, ImgCDL 97.0/97.0, and ConsCDL 91.0/90.3, better than prior CDL generation approaches; the refined data improves results further.

Methodology in Plain English

The method splits the geometry problem into two roles.

First, an MLLM Interpreter looks at the diagram plus the problem text and writes out a Conditional Declaration Language (CDL) description. CDL has three parts: ConsCDL (the basic construction of the diagram, such as shapes, collinear, and cocircular facts), ImgCDL (geometric and algebraic relations read off the diagram), and TextCDL (relations stated in the problem text). CDL is structured and concise, unlike free-form captions.

Second, an off-the-shelf LLM Solver (Qwen3 30B) receives only the generated CDL — without the CoT — and reasons to the final answer. The solver is always kept frozen.

Training the interpreter happens in two stages. In stage one, supervised fine-tuning minimizes the negative log-likelihood of the reasoning steps and the final description, using CoT-augmented data wrapped in <think>...</think> and <cdl>...</cdl> tags, where the CoTs were auto-generated by a Python parser. In stage two, GRPO samples a group of candidate responses, normalizes their scores into advantages, and updates the policy while penalizing deviation from a reference model.

The distinctive reward design compares the generated CDL piece-by-piece with the ground-truth CDL using greedy matching, then scores recall (completeness) and precision (penalizing inaccuracy and repetition); the ConsCDL reward is the average of the two, and ImgCDL and TextCDL rewards are computed the same way. A binary format reward checks that the tags are respected. The rewards are combined with weights α = 0.1 and γ = 0.5.

The dataset pipeline started from Formalgeo7k v2, which contains 7,000 geometric question-answer pairs with diagrams and CDL annotations. After manual review and CoT incorporation the result is Formalgeo7k-Rec-CoT, split 0.8/0.2 into 5,550 training pairs and 1,390 validation pairs.

Why This Matters

The paper challenges the dominant assumption that geometry solving requires fine-tuning one large multimodal model end-to-end. By fixing the interface between seeing and reasoning as a concise, machine-checkable formal language, it keeps a strong LLM's reasoning intact, uses far less training data, and makes the reward signal dense enough to train with reinforcement learning. The appendix also provides a formal argument that CDL's description space is much smaller than that of general text, because shape descriptions are restricted, each relation has exactly one possible CDL form, and CDL contains no irrelevant words.

Potential real-world applications:

• Automated geometry tutoring that explains and checks a student's diagram interpretation step by step.
• Automated grading of handwritten or textbook geometry problems where only the answer and the diagram are available.
• Math-education content pipelines that convert diagrams into structured, searchable formal annotations.
• Assistive tools that verbalize geometry figures for learners who cannot rely on the visual channel.

Industry relevance: the recipe — a small trained perception model plus a frozen reasoning LLM — is attractive for teams that cannot afford large-scale multimodal fine-tuning, want to swap in newer reasoning models without retraining perception, or need audit-friendly intermediate representations for education and assessment products. The paper also reports that this generalization holds across multiple LLM solvers, which supports deployment flexibility.

Future Directions

• Extending to other formal domains. The paper's conclusion hopes the work inspires research on how to better exploit the inherent reasoning capability of LLMs for complex multimodal reasoning tasks beyond plane geometry.
• Improving ImgCDL and TextCDL generation. CoTs for these two CDL types degraded performance in this study, and the authors hypothesize that they rely more on direct perception; how to extract reliable diagram relations remains open.
• Replacing manual review. The dataset depended on four qualified annotators and a multi-stage review process; scaling this quality control without human labor is unresolved.
• Choosing or combining rewards. The paper shows that adding a solution-based reward hurt; understanding when end-task rewards help versus hurt relative to structured matching rewards is still an open design question.

Target Audience

Researchers and practitioners working on multimodal reasoning, mathematical problem solving, and reinforcement learning for language models; engineers building educational AI or diagram-understanding systems; and anyone interested in formal intermediate representations that let a frozen LLM perform the reasoning while a smaller model handles perception.

Authors’ abstract

Plane Geometry Problem Solving (PGPS) is a multimodal reasoning task that aims to solve a plane geometric problem based on a geometric diagram and problem textual descriptions. Although Large Language Models (LLMs) possess strong reasoning skills, their direct application to PGPS is hindered by their inability to process visual diagrams. Existing works typically fine-tune Multimodal LLMs (MLLMs) end-to-end on large-scale PGPS data to enhance visual understanding and reasoning simultaneously. However, such joint optimization may compromise base LLMs' inherent reasoning capability. In this work, we observe that LLM itself is potentially a powerful PGPS solver when appropriately formulating visual information as textual descriptions. We propose to train a MLLM Interpreter to generate geometric descriptions for the visual diagram, and an off-the-shelf LLM is utilized to perform reasoning. Specifically, we choose Conditional Declaration Language (CDL) as the geometric description as its conciseness eases the MLLM Interpreter training. The MLLM Interpreter is fine-tuned via CoT (Chain-of-Thought)-augmented SFT followed by GRPO to generate CDL. Instead of using a conventional solution-based reward that compares the reasoning result with the ground-truth answer, we design CDL matching rewards to facilitate more effective GRPO training, which provides more direct and denser guidance for CDL generation. To support training, we construct a new dataset, Formalgeo7k-Rec-CoT, by manually reviewing Formalgeo7k v2 and incorporating CoT annotations. Extensive experiments on Formalgeo7k-Rec-CoT, Unigeo, and MathVista show our method (finetuned on only 5.5k data) performs favorably against leading open-source and closed-source MLLMs.