# DiagramIR: An Automatic Pipeline for Educational Math Diagram Evaluation

arXiv: https://arxiv.org/abs/2511.08283 · HTML: https://mlsanity.com/paper/2511.08283/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 37 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Builds on (most cited first, 7 of 7)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?](https://mlsanity.com/paper/2403.14624/) (2024)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [From Text to Visuals: Using LLMs to Generate Math Diagrams with Vector Graphics](https://mlsanity.com/paper/2503.07429/) (2025)
- ["The teachers are confused as well": A Multiple-Stakeholder Ethics Discussion on Large Language Models in Computing Education](https://mlsanity.com/paper/2401.12453/) (2024)

## Built on by (most cited first, 3 of 3)

- [How RL Unlocks the Aha Moment in Geometric Interleaved Reasoning](https://mlsanity.com/paper/2603.01070/) (2026)
- [EduIllustrate: Towards Scalable Automated Generation Of Multimodal Educational Content](https://mlsanity.com/paper/2604.05005/) (2026)
- [Math-Vision Diagrams: A Comprehensive Benchmark for Evaluating LLM Mathematical Diagram Generation Capabilities](https://mlsanity.com/paper/2608.08964/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly being adopted as tools for learning; however, most tools remain text-only, limiting their usefulness for domains where visualizations are essential, such as mathematics. Recent work shows that LLMs are capable of generating code that compiles to educational figures, but a major bottleneck remains: scalable evaluation of these diagrams. We address this by proposing DiagramIR: an automatic and scalable evaluation pipeline for geometric figures. Our method relies on intermediate representations (IRs) of LaTeX TikZ code. We compare our pipeline to other evaluation baselines such as LLM-as-a-Judge, showing that our approach has higher agreement with human raters. This evaluation approach also enables smaller models like GPT-4.1-Mini to perform comparably to larger models such as GPT-5 at a 10x lower inference cost, which is important for deploying accessible and scalable education technologies.