mlsanityData to 14 Oct 2026

Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models

arXiv 2406.17294 ↗First posted 25 Jun 2024cs.CLcited by 133

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Large language models (LLMs) have demonstrated impressive reasoning capabilities, particularly in textual mathematical problem-solving. However, existing open-source image instruction fine-tuning datasets, containing limited question-answer pairs per image, do not fully exploit visual information to enhance the multimodal mathematical reasoning capabilities of Multimodal LLMs (MLLMs). To bridge this gap, we address the lack of high-quality, diverse multimodal mathematical datasets by collecting 40K high-quality images with question-answer pairs from 24 existing datasets and synthesizing 320K new pairs, creating the MathV360K dataset, which enhances both the breadth and depth of multimodal mathematical questions. We introduce Math-LLaVA, a LLaVA-1.5-based model fine-tuned with MathV360K. This novel approach significantly improves the multimodal mathematical reasoning capabilities of LLaVA-1.5, achieving a 19-point increase and comparable performance to GPT-4V on MathVista's minitest split, and yielding leading performance on Math-V and MathVerse. Furthermore, Math-LLaVA demonstrates enhanced generalizability, showing substantial improvements on the MMMU benchmark. Our research highlights the importance of dataset diversity and synthesis in advancing MLLMs' mathematical reasoning abilities. The code and data are available at: urlhttps://github.com/HZQ950419/Math-LLaVA.

Key facts. Cited by 133 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 66 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

38 of 66 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  2. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models25 Oct
  3. 2023T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answering5 May
  4. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model28 Apr
  5. 2023Logical Implications for Visual Question Answering Consistency16 Mar
  6. 2022Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning1 Dec
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep
  8. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoning10 Aug
  9. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  8. 2015VQA: Visual Question Answeringcited by 2,506
  9. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  10. 2019Towards VQA Models That Can Readcited by 1,718
  11. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  12. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
17 more
  1. 2016A Diagram Is Worth A Dozen Imagescited by 804
  2. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  3. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  4. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  5. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  6. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  7. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 383
  8. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 350
  9. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 340
  10. 2019PlotQA: Reasoning over Scientific Plotscited by 321
  11. 2017FigureQA: An Annotated Figure Dataset for Visual Reasoningcited by 319
  12. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 274
  13. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 267
  14. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 210
  15. 2022UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expressioncited by 137
  16. 2022MapQA: A Dataset for Question Answering on Choropleth Mapscited by 85
  17. 2023Non-Autoregressive Math Word Problem Solver with Unified Tree Structurecited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 19; 2025: 86; 2026: 28.

133 papers.

Sort:

By area:

  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  2. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  3. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  4. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 248
  5. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
  6. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 124
  7. 2024MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Enginecited by 113
  8. 2025Baichuan-Omni-1.5 Technical Reportcited by 111
  9. 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Modelscited by 99
  10. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  11. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learningcited by 71
  12. 2024MultiMath: Bridging Visual and Mathematical Reasoning for Large Language Modelscited by 70
38 more
  1. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Modelcited by 65
  2. 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learningcited by 57
  3. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  4. 2025FineVision: Open Data Is All You Needcited by 53
  5. 2025MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoningcited by 49
  6. 2025MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoningcited by 45
  7. 2025SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Rewardcited by 45
  8. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Rewardcited by 45
  9. 2025Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuningcited by 44
  10. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Startcited by 41
  11. 2025MM-Verify: Enhancing Multimodal Reasoning with Chain-of-Thought Verificationcited by 39
  12. 2024ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detectioncited by 37
  13. 2024InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoningcited by 36
  14. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 36
  15. 2025VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selectioncited by 34
  16. 2025First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-Trainingcited by 34
  17. 2024Theorem-Validated Reverse Chain-of-Thought Problem Generation for Geometric Reasoningcited by 32
  18. 2024Diving into Self-Evolving Training for Multimodal Reasoningcited by 32
  19. 2025MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contextscited by 32
  20. 2025Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoningcited by 32
  21. 2025Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoningcited by 32
  22. 2025We-Math 2.0: A Versatile MathBook System for Incentivizing Visual Mathematical Reasoningcited by 32
  23. 2025MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable Step-Level Supervisioncited by 31
  24. 2024GalleryGPT: Analyzing Paintings with Large Multimodal Modelscited by 28
  25. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  26. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  27. 2025V-Thinker: Interactive Thinking with Imagescited by 26
  28. 2025AStar: Boosting Multimodal Reasoning with Automated Structured Thinkingcited by 25
  29. 2024GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-trainingcited by 23
  30. 2025MLLM-CL: Continual Learning for Multimodal Large Language Modelscited by 23
  31. 2025CodePlot-CoT: Mathematical Visual Reasoning by Thinking with Code-Driven Imagescited by 23
  32. 2025MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in Videoscited by 22
  33. 2025Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weightscited by 20
  34. 2025M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performancecited by 19
  35. 2024Leopard: A Vision Language Model For Text-Rich Multi-Image Taskscited by 18
  36. 2025Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural Integrationcited by 18
  37. 2025VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement Learningcited by 18
  38. 2024POINTS: Improving Your Vision-language Model with Affordable Strategiescited by 17

50 most cited of 133 All 133 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models
  2. 2024Improving Multimodal LLMs Ability In Geometry Problem Solving, Reasoning, And Multistep Scoring
  3. 2025MathSE: Improving Multimodal Mathematical Reasoning via Self-Evolving Iterative Reflection and Reward-Guided Fine-Tuning
  4. 2024MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Model
  5. 2026M3Kang: Evaluating Multilingual Multimodal Mathematical Reasoning in Vision-Language Models
  6. 2024Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models
  7. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges
  8. 2025Integrating Visual Interpretation and Linguistic Reasoning for Math Problem Solving