# Math Blind: Failures in Diagram Understanding Undermine Reasoning in MLLMs

arXiv: https://arxiv.org/abs/2503.20745 · HTML: https://mlsanity.com/paper/2503.20745/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 20 of its 62 references that are in the map.
- First posted on arXiv 26 Mar 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 20 of 20)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://mlsanity.com/paper/1909.02164/) (2019)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model](https://mlsanity.com/paper/2401.16420/) (2024)
- [FigureQA: An Annotated Figure Dataset for Visual Reasoning](https://mlsanity.com/paper/1710.07300/) (2017)
- [G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model](https://mlsanity.com/paper/2312.11370/) (2023)
- [Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models](https://mlsanity.com/paper/2406.17294/) (2024)
- [MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine](https://mlsanity.com/paper/2407.08739/) (2024)
- [MultiMath: Bridging Visual and Mathematical Reasoning for Large Language Models](https://mlsanity.com/paper/2409.00147/) (2024)
- [Unlocking Multimodal Mathematical Reasoning via Process Reward Model](https://mlsanity.com/paper/2501.04686/) (2025)
- [Theorem-Validated Reverse Chain-of-Thought Problem Generation for Geometric Reasoning](https://mlsanity.com/paper/2410.17885/) (2024)
- [AutoGeo: Automating Geometric Image Dataset Creation for Enhanced Geometry Understanding](https://mlsanity.com/paper/2409.09039/) (2024)
- [Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs](https://mlsanity.com/paper/2501.06430/) (2025)
- [Improving Multi-modal Large Language Model through Boosting Vision Capabilities](https://mlsanity.com/paper/2410.13733/) (2024)
- [Reasoning with Graphs: Structuring Implicit Knowledge to Enhance LLMs Reasoning](https://mlsanity.com/paper/2501.07845/) (2025)

## Built on by (most cited first, 16 of 16)

- [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025)
- [CountQA: How Well Do MLLMs Count in the Wild?](https://mlsanity.com/paper/2508.06585/) (2025)
- [VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMs](https://mlsanity.com/paper/2506.06727/) (2025)
- [Generalizable Geometric Image Caption Synthesis](https://mlsanity.com/paper/2509.15217/) (2025)
- [A Survey of Deep Learning for Geometry Problem Solving](https://mlsanity.com/paper/2507.11936/) (2025)
- [GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoning](https://mlsanity.com/paper/2603.22687/) (2026)
- [Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams](https://mlsanity.com/paper/2608.12262/) (2026)
- [Hierarchically Robust Zero-shot Vision-language Models](https://mlsanity.com/paper/2604.18867/) (2026)
- [GeoSym127K: Scalable Symbolically-verifiable Synthesis for Multimodal Geometric Reasoning](https://mlsanity.com/paper/2605.16371/) (2026)
- [FractalBench: Diagnosing Visual-Mathematical Reasoning Through Recursive Program Synthesis](https://mlsanity.com/paper/2511.06522/) (2025)
- [Hierarchical Process Reward Models are Symbolic Vision Learners](https://mlsanity.com/paper/2512.03126/) (2025)
- [A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning](https://mlsanity.com/paper/2603.08291/) (2026)
- [Draw2Think: Harnessing Geometry Reasoning through Constraint Engine Interaction](https://mlsanity.com/paper/2605.20743/) (2026)
- [The Image Reconstruction Game: Drawing Common Ground Through Iterative Multimodal Dialogue](https://mlsanity.com/paper/2606.01901/) (2026)
- [SADL: What to Ignore? A Benchmark for Subject-Aware Distractor Localization](https://mlsanity.com/paper/2606.30393/) (2026)
- [Func-R1: Incentivizing Mathematical Function Reasoning in Multimodal Large Language Models](https://mlsanity.com/paper/2609.14779/) (2026)

## Abstract (arXiv)

Diagrams represent a form of visual language that encodes abstract concepts and relationships through structured symbols and their spatial arrangements. Unlike natural images, they are inherently symbolic, and entirely artificial. They thus pose unique challenges for Multimodal Large Language Models (MLLMs) distinct from natural image processing. Recent studies have shown that MLLMs often exhibit flawed reasoning and hallucinations when handling diagram inputs. We investigate here whether these limitations stem from shortcomings in the models' ability to interpret diagrams themselves. To this end, we develop a diagnostic test suite that isolates perception from reasoning. Our systematic evaluation reveals that MLLMs perform poorly on basic perceptual tasks, e.g., shape classification, object counting, relationship identification, and object grounding, with near-zero accuracy on fine-grained grounding. Further analysis shows that weak diagram perception leads to "blind faith in text", where models rely on textual shortcuts rather than visual understanding (that is, they are Math Blind). We hypothesize that enabling models to capture the inherent structural properties of diagrams, represented as graphs of primitives and their interrelationships, is essential for improving diagram understanding. Experiments with 7B and 32B MLLMs validate this assumption, with models trained on such representations achieving a +79% gain on the grounding task. Crucially, these gains transfer to reasoning, achieving 3-4% cross-suite improvements on three public benchmarks even without additional chain-of-thought reasoning data. Our findings demonstrate that low-level perception supports faithful high-level reasoning in mathematical MLLMs. We provide both methodological frameworks and empirical evidence to guide future research in this direction.