mlsanityData to 7 Oct 2026

MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts

arXiv 2310.02255 ↗First posted 3 Oct 2023cs.CV cs.AI cs.CLcited by 1,582

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) and Large Multimodal Models (LMMs) exhibit impressive problem-solving skills in many tasks and domains, but their ability in mathematical reasoning in visual contexts has not been systematically studied. To bridge this gap, we present MathVista, a benchmark designed to combine challenges from diverse mathematical and visual tasks. It consists of 6,141 examples, derived from 28 existing multimodal datasets involving mathematics and 3 newly created datasets (i.e., IQTest, FunctionQA, and PaperQA). Completing these tasks requires fine-grained, deep visual understanding and compositional reasoning, which all state-of-the-art foundation models find challenging. With MathVista, we have conducted a comprehensive, quantitative evaluation of 12 prominent foundation models. The best-performing GPT-4V model achieves an overall accuracy of 49.9%, substantially outperforming Bard, the second-best performer, by 15.1%. Our in-depth analysis reveals that the superiority of GPT-4V is mainly attributed to its enhanced visual perception and mathematical reasoning. However, GPT-4V still falls short of human performance by 10.4%, as it often struggles to understand complex figures and perform rigorous reasoning. This significant gap underscores the critical role that MathVista will play in the development of general-purpose AI agents capable of tackling mathematically intensive and visually rich real-world tasks. We further explore the new ability of self-verification, the application of self-consistency, and the interactive chatbot capabilities of GPT-4V, highlighting its promising potential for future research. The project is available at https://mathvista.github.io/.

Key facts. Cited by 1,582 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 70 of its 96 references that are in the map. First posted on arXiv 3 Oct 2023.

Builds on

70 of 96 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model28 Apr
  2. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering19 Dec
  3. 2022Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning1 Dec
  4. 2022Abstract Visual Reasoning with Tangram Shapes29 Nov
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep
  6. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoning10 Aug
  7. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 63

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2023Let's Verify Step by Stepcited by 3,598
38 more
  1. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  5. 2015VQA: Visual Question Answeringcited by 2,489
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  8. 2019Towards VQA Models That Can Readcited by 1,696
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  10. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  11. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  12. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  13. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  14. 2023BloombergGPT: A Large Language Model for Financecited by 923
  15. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  16. 2016A Diagram Is Worth A Dozen Imagescited by 793
  17. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  18. 2022Galactica: A Large Language Model for Sciencecited by 736
  19. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  20. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  21. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  22. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  23. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  24. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  25. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  26. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 376
  27. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  28. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  29. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  30. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  31. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  32. 2019PlotQA: Reasoning over Scientific Plotscited by 317
  33. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 314
  34. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 272
  35. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 262
  36. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  37. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 246
  38. 2023FinGPT: Open-Source Financial Large Language Modelscited by 244

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 24; 2024: 293; 2025: 678; 2026: 587.

1,582 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  3. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  5. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  6. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  7. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  8. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  9. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  10. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  11. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  12. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
38 more
  1. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  2. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  3. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  4. 2025Qwen2.5-Omni Technical Reportcited by 704
  5. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  6. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  7. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  8. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
  9. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  10. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  11. 2025Humanity's Last Examcited by 467
  12. 2025Qwen3-Omni Technical Reportcited by 451
  13. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  14. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  15. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  16. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  17. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
  18. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  19. 2025Kimi-VL Technical Reportcited by 381
  20. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
  21. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  22. 2024Mini-Gemini: Mining the Potential of Multi-modality Vision Language Modelscited by 361
  23. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  24. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  25. 2024What matters when building vision-language models?cited by 321
  26. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  27. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  28. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 315
  29. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
  30. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 294
  31. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 283
  32. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
  33. 2024MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmarkcited by 281
  34. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
  35. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 267
  36. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 244
  37. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 238
  38. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233

50 most cited of 1,582 All 1,582 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset
  2. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
  3. 2025The Role of Visual Modality in Multimodal Mathematical Reasoning: Challenges and Insights
  4. 2025MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models
  5. 2025Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs
  6. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts
  7. 2025GeoThought: A Dataset for Enhancing Mathematical Geometry Reasoning in Vision-Language Models
  8. 2024Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models