mlsanityData to 14 Oct 2026

Diagram Formalization Enhanced Multi-Modal Geometry Problem Solver

arXiv 2409.04214 ↗First posted 6 Sep 2024cs.CVcited by 11

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Mathematical reasoning remains an ongoing challenge for AI models, especially for geometry problems that require both linguistic and visual signals. As the vision encoders of most MLLMs are trained on natural scenes, they often struggle to understand geometric diagrams, performing no better in geometry problem solving than LLMs that only process text. This limitation is amplified by the lack of effective methods for representing geometric relationships. To address these issues, we introduce the Diagram Formalization Enhanced Geometry Problem Solver (DFE-GPS), a new framework that integrates visual features, geometric formal language, and natural language representations. We propose a novel synthetic data approach and create a large-scale geometric dataset, SynthGeo228K, annotated with both formal and natural language captions, designed to enhance the vision encoder for a better understanding of geometric structures. Our framework improves MLLMs' ability to process geometric diagrams and extends their application to open-ended tasks on the formalgeo7k dataset.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 24 of its 34 references that are in the map. First posted on arXiv 6 Sep 2024.

Builds on

24 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagram22 Feb
  2. 2022UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression6 Dec

Other references · 22

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  7. 2024Qwen2 Technical Reportcited by 2,120
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  10. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  11. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  12. 2024Yi: Open Foundation Models by 01.AIcited by 719
10 more
  1. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  2. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  3. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  4. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 383
  5. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 228
  6. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 210
  7. 2024DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchcited by 197
  8. 2024GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solvingcited by 45
  9. 2024GeoGPT4V: Towards Geometric Multi-modal Large Language Models with Geometric Image Generationcited by 32
  10. 2020Automatically Building Diagrams for Olympiad Geometry Problemscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 5.

11 papers.

  1. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  2. 2025VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selectioncited by 34
  3. 2025GeoUni: A Unified Model for Generating Geometry Diagrams, Problems and Problem Solutionscited by 11
  4. 2025Towards Geometry Problem Solving in the Large Model Era: A Surveycited by 8
  5. 2025A Survey of Deep Learning for Geometry Problem Solvingcited by 4
  6. 2025Foundation of Intelligence: Review of Math Word Problems from Human Cognition Perspectivecited by 1
  7. 2026Enhancing Geometric Perception in VLMs via Translator-Guided Reinforcement Learningcited by 1
  8. 2026Concise Geometric Description as a Bridge: Unleashing the Potential of LLM for Plane Geometry Problem Solvingcited by 0
  9. 2026Verifiable Geometry Problem Solving: Solver-Driven Autoformalization and Theorem Proposingcited by 0
  10. 2026Euclid-Omni : A Unified Neuro-Symbolic Framework for Plane Geometrycited by 0
  11. 2026GeoReform: Reflective Formalization Evolution for Multimodal Geometry Problem Solvingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training
  2. 2025GeoFM: Enhancing Geometric Reasoning of MLLMs via Synthetic Data Generation through Formal Language
  3. 2025GeoRef: Referring Expressions in Geometry via Task Formulation, Synthetic Supervision, and Reinforced MLLM-based Solutions
  4. 2024EAGLE: Elevating Geometric Reasoning through LLM-empowered Visual Instruction Tuning
  5. 2026Geoparsing: Diagram Parsing for Plane and Solid Geometry with a Unified Formal Language
  6. 2026From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning
  7. 2025Bridging Formal Language with Chain-of-Thought Reasoning to Geometry Problem Solving
  8. 2026FormalAnalyticGeo: A Neural-Symbolic Based Framework for Multimodal Analytic Geometry Problem Generation