mlsanityData to 7 Oct 2026

MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts

arXiv 2502.20808 ↗First posted 28 Feb 2025cs.AIcited by 32

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have shown promising capabilities in mathematical reasoning within visual contexts across various datasets. However, most existing multimodal math benchmarks are limited to single-visual contexts, which diverges from the multi-visual scenarios commonly encountered in real-world mathematical applications. To address this gap, we introduce MV-MATH: a meticulously curated dataset of 2,009 high-quality mathematical problems. Each problem integrates multiple images interleaved with text, derived from authentic K-12 scenarios, and enriched with detailed annotations. MV-MATH includes multiple-choice, free-form, and multi-step questions, covering 11 subject areas across 3 difficulty levels, and serves as a comprehensive and rigorous benchmark for assessing MLLMs' mathematical reasoning in multi-visual contexts. Through extensive experimentation, we observe that MLLMs encounter substantial challenges in multi-visual math tasks, with a considerable performance gap relative to human capabilities on MV-MATH. Furthermore, we analyze the performance and error patterns of various models, providing insights into MLLMs' mathematical reasoning capabilities within multi-visual settings.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 44 of its 56 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

44 of 56 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024CMM-Math: A Chinese Multimodal Math Dataset To Evaluate and Enhance the Mathematics Reasoning of Large Multimodal Models4 Sep
  2. 2024MultiMath: Bridging Visual and Mathematical Reasoning for Large Language Models30 Aug
  3. 2024Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models25 Jun
  4. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  5. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences19 Jan
  6. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct

Other references · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  11. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  12. 2023A Survey of Large Language Modelscited by 2,537
26 more
  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  3. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  5. 2023VILA: On Pre-training for Visual Language Modelscited by 686
  6. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  7. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  8. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  9. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  10. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  11. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  12. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 224
  13. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  14. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 208
  15. 2023Multimodal Large Language Models: A Surveycited by 193
  16. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 192
  17. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 178
  18. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  19. 2024MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmarkcited by 103
  20. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 76
  21. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 73
  22. 2025Distilling Multi-modal Large Language Models for Autonomous Drivingcited by 46
  23. 2024GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solvingcited by 44
  24. 2023Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Modelscited by 28
  25. 2024CMMaTH: A Chinese Multi-modal Math Skill Evaluation Benchmark for Foundation Modelscited by 15
  26. 2024SuperCLUE-Math6: Graded Multi-Step Math Reasoning Benchmark for LLMs in Chinesecited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 12.

32 papers.

Sort:

By area:

  1. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  2. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
  3. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
  4. 2025VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Modelscited by 31
  5. 2025More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Modelscited by 26
  6. 2025CodePlot-CoT: Mathematical Visual Reasoning by Thinking with Code-Driven Imagescited by 22
  7. 2025a1: Steep Test-time Scaling Law via Environment Augmented Generationcited by 13
  8. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 13
  9. 2025MATP-BENCH: Can MLLM Be a Good Automated Theorem Prover for Multimodal Problems?cited by 13
  10. 2025PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoningcited by 11
  11. 2025Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependencycited by 10
  12. 2025Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesiscited by 8
20 more
  1. 2025VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoningcited by 7
  2. 2025Large Language Models as Computable Approximations to Solomonoff Inductioncited by 6
  3. 2025VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMscited by 6
  4. 2025GeoLaux: A Benchmark for Evaluating MLLMs' Geometry Performance on Long-Step Problems Requiring Auxiliary Linescited by 6
  5. 2025AIM-Bench: Evaluating Decision-making Biases of Agentic LLM as Inventory Managercited by 6
  6. 2025GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Modelscited by 6
  7. 2025A Survey of Deep Learning for Geometry Problem Solvingcited by 4
  8. 2026ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answeringcited by 3
  9. 2026VisTIRA: Closing the Image-Text Modality Gap in Visual Math Reasoning via Structured Tool Integrationcited by 2
  10. 2026TPRU: Advancing Temporal and Procedural Understanding in Large Multimodal Modelscited by 2
  11. 2026Geoparsing: Diagram Parsing for Plane and Solid Geometry with a Unified Formal Languagecited by 1
  12. 2026DR-MMSearchAgent: Deepening Reasoning in Multimodal Search Agentscited by 1
  13. 2025PyFi: Toward Pyramid-like Financial Image Understanding for VLMs via Adversarial Agentscited by 0
  14. 2026LatentGeo: Learnable Auxiliary Constructions in Latent Space for Multimodal Geometric Reasoningcited by 0
  15. 2026VAMPS: Visual-Assisted Mathematical Problem Solving Benchmarkcited by 0
  16. 2026BiNSGPS: Geometry Problem Solving via Bidirectional Neuro-Symbolic Interactioncited by 0
  17. 2026VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instructcited by 0
  18. 2026DisciplineGen-1M: A Large-Scale Dataset for Multidisciplinary Visual Generation and Editingcited by 0
  19. 2026Math-Vision Diagrams: A Comprehensive Benchmark for Evaluating LLM Mathematical Diagram Generation Capabilitiescited by 0
  20. 2026From Atomic to Agentic: Towards Interpretable Evaluation of LLMs' Agentic Mathematical Capabilitiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MathScape: Benchmarking Multimodal Large Language Models in Real-World Mathematical Contexts
  2. 2025MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models
  3. 2026M3Kang: Evaluating Multilingual Multimodal Mathematical Reasoning in Vision-Language Models
  4. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?
  5. 2025Evaluating Visual Mathematics in Multimodal LLMs: A Multilingual Benchmark Based on the Kangaroo Tests
  6. 2026A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning
  7. 2024MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classification
  8. 2025The Role of Visual Modality in Multimodal Mathematical Reasoning: Challenges and Insights