mlsanityData to 7 Oct 2026

We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?

arXiv 2407.01284 ↗First posted 1 Jul 2024cs.AI cs.CL cs.CVcited by 281

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Visual mathematical reasoning, as a fundamental visual reasoning ability, has received widespread attention from the Large Multimodal Models (LMMs) community. Existing benchmarks, such as MathVista and MathVerse, focus more on the result-oriented performance but neglect the underlying principles in knowledge acquisition and generalization. Inspired by human-like mathematical reasoning, we introduce WE-MATH, the first benchmark specifically designed to explore the problem-solving principles beyond end-to-end performance. We meticulously collect and categorize 6.5K visual math problems, spanning 67 hierarchical knowledge concepts and five layers of knowledge granularity. We decompose composite problems into sub-problems according to the required knowledge concepts and introduce a novel four-dimensional metric, namely Insufficient Knowledge (IK), Inadequate Generalization (IG), Complete Mastery (CM), and Rote Memorization (RM), to hierarchically assess inherent issues in LMMs' reasoning process. With WE-MATH, we conduct a thorough evaluation of existing LMMs in visual mathematical reasoning and reveal a negative correlation between solving steps and problem-specific performance. We confirm the IK issue of LMMs can be effectively improved via knowledge augmentation strategies. More notably, the primary challenge of GPT-4o has significantly transitioned from IK to IG, establishing it as the first LMM advancing towards the knowledge generalization stage. In contrast, other LMMs exhibit a marked inclination towards Rote Memorization - they correctly solve composite problems involving multiple knowledge concepts yet fail to answer sub-problems. We anticipate that WE-MATH will open new pathways for advancements in visual mathematical reasoning for LMMs. The WE-MATH data and evaluation code are available at https://github.com/We-Math/We-Math.

Key facts. Cited by 281 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 69 of its 93 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

69 of 93 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Large Multimodal Agents: A Survey23 Feb
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  4. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model28 Apr
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep

Other references · 64

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  12. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
38 more
  1. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  4. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  5. 2015VQA: Visual Question Answeringcited by 2,489
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  8. 2019Towards VQA Models That Can Readcited by 1,696
  9. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  10. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  12. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  13. 2023PaLM 2 Technical Reportcited by 1,154
  14. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  15. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  16. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  17. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  18. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  19. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  20. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  21. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  22. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  23. 2022PAL: Program-aided Language Modelscited by 635
  24. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  25. 2018nocaps: novel object captioning at scalecited by 444
  26. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  27. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 376
  28. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 330
  29. 2019Scene Text Visual Question Answeringcited by 327
  30. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  31. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  32. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 224
  33. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 215
  34. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 208
  35. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 192
  36. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
  37. 2022UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expressioncited by 137
  38. 2023Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Modelscited by 107

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 137; 2026: 127.

281 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  4. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  5. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  6. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  7. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  8. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
  9. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 294
  10. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 238
  11. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
  12. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
38 more
  1. 2025Agentic Reinforced Policy Optimizationcited by 166
  2. 2025Thyme: Think Beyond Imagescited by 160
  3. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoningcited by 134
  4. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
  5. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121
  6. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
  7. 2024MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Enginecited by 111
  8. 2026Self-Distilled RLVRcited by 109
  9. 2025MiMo-VL Technical Reportcited by 106
  10. 2025Ovis2.5 Technical Reportcited by 103
  11. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Modelscited by 102
  12. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 95
  13. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  14. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
  15. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  16. 2026Scaling Reasoning Efficiently via Relaxed On-Policy Distillationcited by 71
  17. 2025Kwai Keye-VL 1.5 Technical Reportcited by 68
  18. 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokenscited by 65
  19. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Modelcited by 64
  20. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 54
  21. 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learningcited by 53
  22. 2024Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoningcited by 52
  23. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learningcited by 51
  24. 2025Kwai Keye-VL Technical Reportcited by 47
  25. 2025MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoningcited by 45
  26. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Rewardcited by 45
  27. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
  28. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipecited by 43
  29. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Startcited by 41
  30. 2025Agentic Entropy-Balanced Policy Optimizationcited by 39
  31. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  32. 2025Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Playcited by 37
  33. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 37
  34. 2025OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipecited by 37
  35. 2025VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Searchcited by 36
  36. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  37. 2025MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoningcited by 35
  38. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 35

50 most cited of 281 All 281 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset
  2. 2025MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts
  3. 2024MathScape: Benchmarking Multimodal Large Language Models in Real-World Mathematical Contexts
  4. 2025MathSight: A Benchmark Exploring Have Vision-Language Models Really Seen in University-Level Mathematical Reasoning?
  5. 2025Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency
  6. 2025VideoMathQA: Benchmarking Mathematical Reasoning via Multimodal Understanding in Videos
  7. 2025MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models
  8. 2024MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classification