# GeoGPT4V: Towards Geometric Multi-modal Large Language Models with Geometric Image Generation

arXiv: https://arxiv.org/abs/2406.11503 · HTML: https://mlsanity.com/paper/2406.11503/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 27 of 27)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://mlsanity.com/paper/2311.12793/) (2023)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023)
- [DeepSeek-VL: Towards Real-World Vision-Language Understanding](https://mlsanity.com/paper/2403.05525/) (2024)
- [FILIP: Fine-grained Interactive Language-Image Pre-Training](https://mlsanity.com/paper/2111.07783/) (2021)
- [WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct](https://mlsanity.com/paper/2308.09583/) (2023)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://mlsanity.com/paper/2402.14804/) (2024)
- [Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning](https://mlsanity.com/paper/2105.04165/) (2021)
- [Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models](https://mlsanity.com/paper/2311.06607/) (2023)
- [InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model](https://mlsanity.com/paper/2401.16420/) (2024)
- [GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning](https://mlsanity.com/paper/2105.14517/) (2021)
- [Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification](https://mlsanity.com/paper/2308.07921/) (2023)
- [MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning](https://mlsanity.com/paper/2310.03731/) (2023)
- [OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset](https://mlsanity.com/paper/2402.10176/) (2024)
- [UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression](https://mlsanity.com/paper/2212.02746/) (2022)
- [Automatic Model Selection with Large Language Models for Reasoning](https://mlsanity.com/paper/2305.14333/) (2023)

## Built on by (most cited first, 30 of 32)

- [Baichuan-Omni-1.5 Technical Report](https://mlsanity.com/paper/2501.15368/) (2025)
- [Unlocking Multimodal Mathematical Reasoning via Process Reward Model](https://mlsanity.com/paper/2501.04686/) (2025)
- [Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoning](https://mlsanity.com/paper/2408.08640/) (2024)
- [MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning](https://mlsanity.com/paper/2505.10557/) (2025)
- [MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Model](https://mlsanity.com/paper/2409.13729/) (2024)
- [M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performance](https://mlsanity.com/paper/2502.18778/) (2025)
- [Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural Integration](https://mlsanity.com/paper/2504.12773/) (2025)
- [Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs](https://mlsanity.com/paper/2501.06430/) (2025)
- [DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoning](https://mlsanity.com/paper/2509.25866/) (2025)
- [Skip-Vision: Efficient and Scalable Acceleration of Vision-Language Models via Adaptive Token Skipping](https://mlsanity.com/paper/2503.21817/) (2025)
- [Diagram Formalization Enhanced Multi-Modal Geometry Problem Solver](https://mlsanity.com/paper/2409.04214/) (2024)
- [GeoUni: A Unified Model for Generating Geometry Diagrams, Problems and Problem Solutions](https://mlsanity.com/paper/2504.10146/) (2025)
- [MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical Problems](https://mlsanity.com/paper/2503.16549/) (2025)
- [Towards Geometry Problem Solving in the Large Model Era: A Survey](https://mlsanity.com/paper/2506.02690/) (2025)
- [From Text to Visuals: Using LLMs to Generate Math Diagrams with Vector Graphics](https://mlsanity.com/paper/2503.07429/) (2025)
- [GeoCoder: Solving Geometry Problems by Generating Modular Code through Vision-Language Models](https://mlsanity.com/paper/2410.13510/) (2024)
- [Generalizable Geometric Image Caption Synthesis](https://mlsanity.com/paper/2509.15217/) (2025)
- [Galaxy Walker: Geometry-aware VLMs For Galaxy-scale Understanding](https://mlsanity.com/paper/2503.18578/) (2025)
- [GeoSDF: Plane Geometry Diagram Synthesis via Signed Distance Field](https://mlsanity.com/paper/2506.13492/) (2025)
- [A Survey of Deep Learning for Geometry Problem Solving](https://mlsanity.com/paper/2507.11936/) (2025)
- [AI's Spatial Intelligence: Evaluating AI's Understanding of Spatial Transformations in PSVT:R and Augmented Reality](https://mlsanity.com/paper/2411.06269/) (2024)
- [MathSE: Improving Multimodal Mathematical Reasoning via Self-Evolving Iterative Reflection and Reward-Guided Fine-Tuning](https://mlsanity.com/paper/2511.06805/) (2025)
- [Med-Scout: Curing MLLMs' Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Training](https://mlsanity.com/paper/2601.23220/) (2026)
- [GeoFocus: Blending Efficient Global-to-Local Perception for Multimodal Geometry Problem-Solving](https://mlsanity.com/paper/2602.08524/) (2026)
- [GeoChallenge: A Multi-Answer Multiple-Choice Benchmark for Geometric Reasoning with Diagrams](https://mlsanity.com/paper/2603.19252/) (2026)
- [GeoSym127K: Scalable Symbolically-verifiable Synthesis for Multimodal Geometric Reasoning](https://mlsanity.com/paper/2605.16371/) (2026)
- [FormalAnalyticGeo: A Neural-Symbolic Based Framework for Multimodal Analytic Geometry Problem Generation](https://mlsanity.com/paper/2607.12982/) (2026)
- [A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning](https://mlsanity.com/paper/2603.08291/) (2026)
- [BiNSGPS: Geometry Problem Solving via Bidirectional Neuro-Symbolic Interaction](https://mlsanity.com/paper/2606.04648/) (2026)
- [VeriGeo: Controllable Geometry Question Generation with Numerical and Analytical Verification](https://mlsanity.com/paper/2606.14176/) (2026)

## Abstract (arXiv)

Large language models have seen widespread adoption in math problem-solving. However, in geometry problems that usually require visual aids for better understanding, even the most advanced multi-modal models currently still face challenges in effectively using image information. High-quality data is crucial for enhancing the geometric capabilities of multi-modal models, yet existing open-source datasets and related efforts are either too challenging for direct model learning or suffer from misalignment between text and images. To overcome this issue, we introduce a novel pipeline that leverages GPT-4 and GPT-4V to generate relatively basic geometry problems with aligned text and images, facilitating model learning. We have produced a dataset of 4.9K geometry problems and combined it with 19K open-source data to form our GeoGPT4V dataset. Experimental results demonstrate that the GeoGPT4V dataset significantly improves the geometry performance of various models on the MathVista and MathVision benchmarks. The code is available at https://github.com/Lanyu0303/GeoGPT4V_Project