# Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams

arXiv: https://arxiv.org/abs/2608.12262 · HTML: https://mlsanity.com/paper/2608.12262/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 18 of its 63 references that are in the map.
- First posted on arXiv 12 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 18 of 18)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [Kimi K2: Open Agentic Intelligence](https://mlsanity.com/paper/2507.20534/) (2025)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [BabyVision: Visual Reasoning Beyond Language](https://mlsanity.com/paper/2601.06521/) (2026)
- [Visual Autoregressive Modeling for Instruction-Guided Image Editing](https://mlsanity.com/paper/2508.15772/) (2025)
- [Math Blind: Failures in Diagram Understanding Undermine Reasoning in MLLMs](https://mlsanity.com/paper/2503.20745/) (2025)
- [VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models](https://mlsanity.com/paper/2508.09945/) (2025)
- [Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of Large Language Model Agents](https://mlsanity.com/paper/2601.12560/) (2026)
- [Agentic Learner with Grow-and-Refine Multimodal Semantic Memory](https://mlsanity.com/paper/2511.21678/) (2025)
- [Human-Centric Foundation Models: Perception, Generation and Agentic Modeling](https://mlsanity.com/paper/2502.08556/) (2025)
- [Enhancing Descriptive Captions with Visual Attributes for Multimodal Perception](https://mlsanity.com/paper/2412.14233/) (2024)
- [WorldVQA: Measuring Atomic World Knowledge in Multimodal Large Language Models](https://mlsanity.com/paper/2602.02537/) (2026)
- [TikZilla: Scaling Text-to-TikZ with High-Quality Data and Reinforcement Learning](https://mlsanity.com/paper/2603.03072/) (2026)
- [ChartE3: A Comprehensive Benchmark for End-to-End Chart Editing](https://mlsanity.com/paper/2601.21694/) (2026)
- [Overcoming Vision Language Model Challenges in Diagram Understanding: A Proof-of-Concept with XML-Driven Large Language Models Solutions](https://mlsanity.com/paper/2502.04389/) (2025)
- [Nodes Are Early, Edges Are Late: Probing Diagram Representations in Large Vision-Language Models](https://mlsanity.com/paper/2603.02865/) (2026)

## Built on by (most cited first, 2 of 2)

- [EdiTikZ: Scientific Figure Editing from Revision Trajectories](https://mlsanity.com/paper/2609.01409/) (2026)
- [MathVis: Can AI Reason through Drawing? A Benchmark for Mathematical Figure Generation](https://mlsanity.com/paper/2610.12954/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration. For example, OpenAI Prism is a free workspace for scientific writing and collaboration. One important feature in Prism is turning scientific diagrams directly into LaTeX TikZ code. In this paper, we build a benchmark, Diagram-MMU, a multi-modal benchmark designed to assess MLLMs' ability for scientific diagram parsing and understanding. Diagram-MMU features 3.7k curated diagrams and 18.3k human-validated questions across six domains. It evaluates MLLMs on three tasks common in vibe writing workspaces: diagram-to-code parsing, diagram-to-code editing, and diagram question answering, alongside agentic settings per task. The evaluation of 12 MLLMs reveals that diagram-to-code tasks are more challenging than diagram question answering: models can reason well over diagrams but struggle to parse and edit them, underscoring the need for methods to enhance MLLMs' capability in diagram-to-code generation. Under agentic settings, most models improve parsing and editing performance but degrade on question answering, while Claude-4.6 Opus consistently improves across all three tasks. Project Page: https://vi-ocean.github.io/projects/diagram-mmu.