# Forgotten Polygons: Multimodal Large Language Models are Shape-Blind

arXiv: https://arxiv.org/abs/2502.15969 · HTML: https://mlsanity.com/paper/2502.15969/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 21 Feb 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 29 of 29)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://mlsanity.com/paper/2501.17161/) (2025)
- [MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?](https://mlsanity.com/paper/2403.14624/) (2024)
- [LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models](https://mlsanity.com/paper/2407.07895/) (2024)
- [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://mlsanity.com/paper/2402.14804/) (2024)
- [Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models](https://mlsanity.com/paper/2409.17146/) (2024)
- [G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model](https://mlsanity.com/paper/2312.11370/) (2023)
- [Improve Vision Language Model Chain-of-thought Reasoning](https://mlsanity.com/paper/2410.16198/) (2024)
- [Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models](https://mlsanity.com/paper/2406.17294/) (2024)
- [MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine](https://mlsanity.com/paper/2407.08739/) (2024)
- [Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought](https://mlsanity.com/paper/2501.04682/) (2025)
- [Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations](https://mlsanity.com/paper/2410.02762/) (2024)
- [Vision-Language Models Do Not Understand Negation](https://mlsanity.com/paper/2501.09425/) (2025)
- [Learning to Compose Soft Prompts for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2204.03574/) (2022)
- [Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoning](https://mlsanity.com/paper/2408.08640/) (2024)
- [What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation](https://mlsanity.com/paper/2406.16320/) (2024)
- [Evaluating Large Vision-and-Language Models on Children's Mathematical Olympiads](https://mlsanity.com/paper/2406.15736/) (2024)
- [Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs](https://mlsanity.com/paper/2501.06430/) (2025)
- [Vision-Language Models Create Cross-Modal Task Representations](https://mlsanity.com/paper/2410.22330/) (2024)

## Built on by (most cited first, 16 of 16)

- [Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfacts](https://mlsanity.com/paper/2505.17127/) (2025)
- [Perception Before Reasoning: Two-Stage Reinforcement Learning for Visual Reasoning in Vision-Language Models](https://mlsanity.com/paper/2509.13031/) (2025)
- [Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs](https://mlsanity.com/paper/2504.15280/) (2025)
- [Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs](https://mlsanity.com/paper/2502.16435/) (2025)
- [Position: Reasoning After Perception Means Reasoning Without Vision](https://mlsanity.com/paper/2507.16863/) (2025)
- [Mechanisms of Prompt-Induced Hallucination in Vision-Language Models](https://mlsanity.com/paper/2601.05201/) (2026)
- [Uncovering Grounding IDs: How External Cues Shape Multimodal Binding](https://mlsanity.com/paper/2509.24072/) (2025)
- [Shape and Texture Recognition in Large Vision-Language Models](https://mlsanity.com/paper/2503.23062/) (2025)
- [CounterCount: A Diagnostic Framework for Counting Bias in Vision Language Models](https://mlsanity.com/paper/2605.17826/) (2026)
- [VESTA: Visual Exploration with Statistical Tool Agents](https://mlsanity.com/paper/2606.00384/) (2026)
- [On Semiotic-Grounded Interpretive Evaluation of Generative Art](https://mlsanity.com/paper/2604.08641/) (2026)
- [Is There Knowledge Left to Extract? Evidence of Fragility in Medically Fine-Tuned Vision-Language Models](https://mlsanity.com/paper/2604.09841/) (2026)
- [Math Takes Two: A test for emergent mathematical reasoning in communication](https://mlsanity.com/paper/2604.21935/) (2026)
- [Multimodal QUD: Inquisitive Questions from Scientific Figures](https://mlsanity.com/paper/2604.23733/) (2026)
- [A Geometric View of Counterfactual Behavior: Interaction of Boundary Proximity and Local Support](https://mlsanity.com/paper/2606.04209/) (2026)
- [Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception](https://mlsanity.com/paper/2606.06714/) (2026)

## Abstract (arXiv)

Despite strong performance on vision-language tasks, Multimodal Large Language Models (MLLMs) struggle with mathematical problem-solving, with both open-source and state-of-the-art models falling short of human performance on visual-math benchmarks. To systematically examine visual-mathematical reasoning in MLLMs, we (1) evaluate their understanding of geometric primitives, (2) test multi-step reasoning, and (3) explore a potential solution to improve visual reasoning capabilities. Our findings reveal fundamental shortcomings in shape recognition, with top models achieving under 50% accuracy in identifying regular polygons. We analyze these failures through the lens of dual-process theory and show that MLLMs rely on System 1 (intuitive, memorized associations) rather than System 2 (deliberate reasoning). Consequently, MLLMs fail to count the sides of both familiar and novel shapes, suggesting they have neither learned the concept of sides nor effectively process visual inputs. Finally, we propose Visually Cued Chain-of-Thought (VC-CoT) prompting, which enhances multi-step mathematical reasoning by explicitly referencing visual annotations in diagrams, boosting GPT-4o's accuracy on an irregular polygon side-counting task from 7% to 93%. Our findings suggest that System 2 reasoning in MLLMs remains an open problem, and visually-guided prompting is essential for successfully engaging visual reasoning. Code available at: https://github.com/rsinghlab/Shape-Blind.