# GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training

arXiv: https://arxiv.org/abs/2412.11863 · HTML: https://mlsanity.com/paper/2412.11863/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 46 of its 62 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 46)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs](https://mlsanity.com/paper/2312.14135/) (2023)
- [Llemma: An Open Language Model For Mathematics](https://mlsanity.com/paper/2310.10631/) (2023)
- [Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning](https://mlsanity.com/paper/2105.04165/) (2021)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Deep Modular Co-Attention Networks for Visual Question Answering](https://mlsanity.com/paper/1906.10770/) (2019)
- [MIMIC-IT: Multi-Modal In-Context Instruction Tuning](https://mlsanity.com/paper/2306.05425/) (2023)
- [MinerU: An Open-Source Solution for Precise Document Content Extraction](https://mlsanity.com/paper/2409.18839/) (2024)
- [MathPrompter: Mathematical Reasoning using Large Language Models](https://mlsanity.com/paper/2303.05398/) (2023)
- [GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning](https://mlsanity.com/paper/2105.14517/) (2021)
- [LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning](https://mlsanity.com/paper/2311.18651/) (2023)
- [G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model](https://mlsanity.com/paper/2312.11370/) (2023)
- [OneLLM: One Framework to Align All Modalities with Language](https://mlsanity.com/paper/2312.03700/) (2023)
- [UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression](https://mlsanity.com/paper/2212.02746/) (2022)

## Built on by (most cited first, 23 of 23)

- [Unlocking Multimodal Mathematical Reasoning via Process Reward Model](https://mlsanity.com/paper/2501.04686/) (2025)
- [RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?](https://mlsanity.com/paper/2501.11284/) (2025)
- [MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs](https://mlsanity.com/paper/2505.21327/) (2025)
- [OmniCaptioner: One Captioner to Rule Them All](https://mlsanity.com/paper/2504.07089/) (2025)
- [Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural Integration](https://mlsanity.com/paper/2504.12773/) (2025)
- [TrustGeoGen: Formal-Verified Data Engine for Trustworthy Multi-modal Geometric Problem Solving](https://mlsanity.com/paper/2504.15780/) (2025)
- [PixelCraft: A Multi-Agent System for High-Fidelity Visual Reasoning on Structured Images](https://mlsanity.com/paper/2509.25185/) (2025)
- [GeoUni: A Unified Model for Generating Geometry Diagrams, Problems and Problem Solutions](https://mlsanity.com/paper/2504.10146/) (2025)
- [Pi-GPS: Enhancing Geometry Problem Solving by Unleashing the Power of Diagrammatic Information](https://mlsanity.com/paper/2503.05543/) (2025)
- [Towards Geometry Problem Solving in the Large Model Era: A Survey](https://mlsanity.com/paper/2506.02690/) (2025)
- [GeoDANO: Geometric VLM with Domain Agnostic Vision Encoder](https://mlsanity.com/paper/2502.11360/) (2025)
- [Empowering Small VLMs to Think with Dynamic Memorization and Exploration](https://mlsanity.com/paper/2506.23061/) (2025)
- [GeoSDF: Plane Geometry Diagram Synthesis via Signed Distance Field](https://mlsanity.com/paper/2506.13492/) (2025)
- [Bridging Formal Language with Chain-of-Thought Reasoning to Geometry Problem Solving](https://mlsanity.com/paper/2508.09099/) (2025)
- [Hilbert-Geo: Solving Solid Geometric Problems by Neural-Symbolic Reasoning](https://mlsanity.com/paper/2605.16385/) (2026)
- [DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometry](https://mlsanity.com/paper/2510.22340/) (2025)
- [GeoFocus: Blending Efficient Global-to-Local Perception for Multimodal Geometry Problem-Solving](https://mlsanity.com/paper/2602.08524/) (2026)
- [Enhancing Geometric Perception in VLMs via Translator-Guided Reinforcement Learning](https://mlsanity.com/paper/2602.22703/) (2026)
- [Non-Parametric Structural Priors for Geometry Theorem Prediction](https://mlsanity.com/paper/2603.04852/) (2026)
- [Geoparsing: Diagram Parsing for Plane and Solid Geometry with a Unified Formal Language](https://mlsanity.com/paper/2604.11600/) (2026)
- [A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning](https://mlsanity.com/paper/2603.08291/) (2026)
- [Euclid-Omni : A Unified Neuro-Symbolic Framework for Plane Geometry](https://mlsanity.com/paper/2608.14585/) (2026)
- [From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning](https://mlsanity.com/paper/2609.10335/) (2026)

## Abstract (arXiv)

Despite their proficiency in general tasks, Multi-modal Large Language Models (MLLMs) struggle with automatic Geometry Problem Solving (GPS), which demands understanding diagrams, interpreting symbols, and performing complex reasoning. This limitation arises from their pre-training on natural images and texts, along with the lack of automated verification in the problem-solving process. Besides, current geometric specialists are limited by their task-specific designs, making them less effective for broader geometric problems. To this end, we present GeoX, a multi-modal large model focusing on geometric understanding and reasoning tasks. Given the significant differences between geometric diagram-symbol and natural image-text, we introduce unimodal pre-training to develop a diagram encoder and symbol decoder, enhancing the understanding of geometric images and corpora. Furthermore, we introduce geometry-language alignment, an effective pre-training paradigm that bridges the modality gap between unimodal geometric experts. We propose a Generator-And-Sampler Transformer (GS-Former) to generate discriminative queries and eliminate uninformative representations from unevenly distributed geometric signals. Finally, GeoX benefits from visual instruction tuning, empowering it to take geometric images and questions as input and generate verifiable solutions. Experiments show that GeoX outperforms both generalists and geometric specialists on publicly recognized benchmarks, such as GeoQA, UniGeo, Geometry3K, and PGPS9k.