# MathVis: Can AI Reason through Drawing? A Benchmark for Mathematical Figure Generation

arXiv: https://arxiv.org/abs/2610.12954 · HTML: https://mlsanity.com/paper/2610.12954/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Mathematical reasoning, one of 17 directions in LLM reasoning.
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Placed in

- LLM reasoning / [Mathematical reasoning](https://mlsanity.com/lr/mathematical-reasoning/)

## Builds on (most cited first, 18 of 18)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?](https://mlsanity.com/paper/2403.14624/) (2024)
- [Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset](https://mlsanity.com/paper/2402.14804/) (2024)
- [Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models](https://mlsanity.com/paper/2406.09403/) (2024)
- [AutomaTikZ: Text-Guided Synthesis of Scientific Vector Graphics with TikZ](https://mlsanity.com/paper/2310.00367/) (2023)
- [Can Large Language Models Understand Symbolic Graphics Programs?](https://mlsanity.com/paper/2408.08313/) (2024)
- [AutoGeo: Automating Geometric Image Dataset Creation for Enhanced Geometry Understanding](https://mlsanity.com/paper/2409.09039/) (2024)
- [TheoremExplainAgent: Towards Video-based Multimodal Explanations for LLM Theorem Understanding](https://mlsanity.com/paper/2502.19400/) (2025)
- [TikZero: Zero-Shot Text-Guided Graphics Program Synthesis](https://mlsanity.com/paper/2503.11509/) (2025)
- [SridBench: Benchmark of Scientific Research Illustration Drawing of Image Generation Model](https://mlsanity.com/paper/2505.22126/) (2025)
- [MagicGeo: Training-Free Text-Guided Geometric Diagram Generation](https://mlsanity.com/paper/2502.13855/) (2025)
- [From Text to Visuals: Using LLMs to Generate Math Diagrams with Vector Graphics](https://mlsanity.com/paper/2503.07429/) (2025)
- [Generating Pedagogically Meaningful Visuals for Math Word Problems: A New Benchmark and Analysis of Text-to-Image Models](https://mlsanity.com/paper/2506.03735/) (2025)
- [GeoLoom: High-quality Geometric Diagram Generation from Textual Input](https://mlsanity.com/paper/2512.08180/) (2025)
- [Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams](https://mlsanity.com/paper/2608.12262/) (2026)
- [MathGen: Revealing the Illusion of Mathematical Competence through Text-to-Image Generation](https://mlsanity.com/paper/2603.27959/) (2026)
- [Math-Vision Diagrams: A Comprehensive Benchmark for Evaluating LLM Mathematical Diagram Generation Capabilities](https://mlsanity.com/paper/2608.08964/) (2026)

## Abstract (arXiv)

This paper introduces MathVis, a benchmark designed to evaluate LLMs' ability to visualize frontier mathematical problems and perform diagram-assisted reasoning. The benchmark consists of 512 tasks, each comprising a mathematical statement, a reference diagram, executable TikZ or SVG code, a Gold Intermediate Representation (IR), atomic rubrics, and error codes. Through rigorous evaluation, GPT-6-Astra, Claude Fable 5, and Gemini 3.1 Pro Preview achieve pass@1 scores of 31.25%, 26.56%, and 22.85%, respectively. We found that although the tasks require visualization, LLMs produce outputs dominated by formal languages, symbols, and graphics programs, rather than explicit diagram-mediated reasoning. Furthermore, while the solutions generated by LLMs may contain all the necessary elements, they are often visually confusing to humans. Ultimately, this benchmark aims to visualize mathematical problems and promote research on enabling LLMs to simplify their reasoning via visual diagrams.