MathVis: Can AI Reason through Drawing? A Benchmark for Mathematical Figure Generation
On the map: LLM reasoning → Mathematical reasoning
Abstract (arXiv)
This paper introduces MathVis, a benchmark designed to evaluate LLMs' ability to visualize frontier mathematical problems and perform diagram-assisted reasoning. The benchmark consists of 512 tasks, each comprising a mathematical statement, a reference diagram, executable TikZ or SVG code, a Gold Intermediate Representation (IR), atomic rubrics, and error codes. Through rigorous evaluation, GPT-6-Astra, Claude Fable 5, and Gemini 3.1 Pro Preview achieve pass@1 scores of 31.25%, 26.56%, and 22.85%, respectively. We found that although the tasks require visualization, LLMs produce outputs dominated by formal languages, symbols, and graphics programs, rather than explicit diagram-mediated reasoning. Furthermore, while the solutions generated by LLMs may contain all the necessary elements, they are often visually confusing to humans. Ultimately, this benchmark aims to visualize mathematical problems and promote research on enabling LLMs to simplify their reasoning via visual diagrams.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 31 references that are in the map. First posted on arXiv 9 Oct 2026.
Builds on
18 of 31 references.
References · 18
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2018Datasheets for Datasetscited by 1,386
- 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
- 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
- 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 268
- 2023AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZcited by 79
- 2024Can Large Language Models Understand Symbolic Graphics Programs?cited by 35
- 2024AutoGeo: Automating Geometric Image Dataset Creation for Enhanced Geometry Understandingcited by 29
- 2025TheoremExplainAgent: Towards Video-based Multimodal Explanations for LLM Theorem Understandingcited by 24
- 2025TikZero: Zero-Shot Text-Guided Graphics Program Synthesiscited by 24
- 2025SridBench: Benchmark of Scientific Research Illustration Drawing of Image Generation Modelcited by 19
- 2025MagicGeo: Training-Free Text-Guided Geometric Diagram Generationcited by 11
6 more
- 2025From Text to Visuals: Using LLMs to Generate Math Diagrams with Vector Graphicscited by 6
- 2025Generating Pedagogically Meaningful Visuals for Math Word Problems: A New Benchmark and Analysis of Text-to-Image Modelscited by 5
- 2025GeoLoom: High-quality Geometric Diagram Generation from Textual Inputcited by 2
- 2026Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagramscited by 2
- 2026MathGen: Revealing the Illusion of Mathematical Competence through Text-to-Image Generationcited by 1
- 2026Math-Vision Diagrams: A Comprehensive Benchmark for Evaluating LLM Mathematical Diagram Generation Capabilitiescited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
- 2026Solving Is Not Drawing: A Benchmark for Diagrammatic Reasoning in Olympiad Geometry
- 2024MathScape: Benchmarking Multimodal Large Language Models in Real-World Mathematical Contexts
- 2024MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine
- 2025CodePlot-CoT: Mathematical Visual Reasoning by Thinking with Code-Driven Images
- 2026Exploring Agentic Workflows for Generating High Quality Math Visual Aids
- 2026Visual Reasoning Benchmark: Evaluating Multimodal LLMs on Classroom-Authentic Visual Problems from Primary Education
- 2025VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMs