Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning
On the map: LLM reasoning → Multimodal reasoning
also in 1 more
Abstract (arXiv)
Scientific reasoning, the process through which humans apply logic, evidence, and critical thinking to explore and interpret scientific phenomena, is essential in advancing knowledge reasoning across diverse fields. However, despite significant progress, current scientific reasoning models still struggle with generalization across domains and often fall short of multimodal perception. Multimodal Large Language Models (MLLMs), which integrate text, images, and other modalities, present an exciting opportunity to overcome these limitations and enhance scientific reasoning. Therefore, this position paper argues that MLLMs can significantly advance scientific reasoning across disciplines such as mathematics, physics, chemistry, and biology. First, we propose a four-stage research roadmap of scientific reasoning capabilities, and highlight the current state of MLLM applications in scientific reasoning, noting their ability to integrate and reason over diverse data types. Second, we summarize the key challenges that remain obstacles to achieving MLLM's full potential. To address these challenges, we propose actionable insights and suggestions for the future. Overall, our work offers a novel perspective on MLLM integration with scientific reasoning, providing the LLM community with a valuable vision for achieving Artificial General Intelligence (AGI).
Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 199 of its 290 references that are in the map. First posted on arXiv 5 Feb 2025.
Builds on
199 of 290 references.
Closest ancestors · 16
Same direction, newest first
- 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs10 Jan
- 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan
- 2025Virgo: A Preliminary Exploration on Reproducing o1-like MLLM3 Jan
- 2024VisScience: An Extensive Benchmark for Evaluating K12 Educational Multi-modal Scientific Reasoning10 Sep
- 2024MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Model10 Sep
- 2024CMM-Math: A Chinese Multimodal Math Dataset To Evaluate and Enhance the Mathematics Reasoning of Large Multimodal Models4 Sep
- 2024Math-PUMA: Progressive Upward Multimodal Alignment to Enhance Mathematical Reasoning16 Aug
- 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?1 Jul
- 2024M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models24 May
- 2024MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classification7 Apr
- 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning10 Jan
- 2023PixelLM: Pixel Reasoning with Large Multimodal Model4 Dec
4 more
- 2023Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Framework24 Jul
- 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
- 2023T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answering5 May
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep
Other references · 183
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2023A Survey of Large Language Modelscited by 2,537
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2024Qwen2 Technical Reportcited by 2,076
38 more
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
- 2024Qwen2.5-Coder Technical Reportcited by 1,151
- 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
- 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
- 2023Large Language Models are not Fair Evaluatorscited by 1,050
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
- 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
- 2023Reasoning with Language Model is Planning with World Modelcited by 846
- 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
- 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
- 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
- 2022Galactica: A Large Language Model for Sciencecited by 736
- 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
- 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
- 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
- 2023DataComp: In search of the next generation of multimodal datasetscited by 635
- 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
- 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
- 2022Towards Reasoning in Large Language Models: A Surveycited by 593
- 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
- 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
- 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 533
- 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
- 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
- 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
- 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
- 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
- 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
- 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
- 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
- 2023Scaling Data-Constrained Language Modelscited by 391
- 2023Llemma: An Open Language Model For Mathematicscited by 385
- 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
50 most cited of 183 All 183 (JSON)
Built on by
Citing papers per year: 2025: 27; 2026: 6.
33 papers.
Sort:
By area:
- 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
- 2025LLM Agents for Education: Advances and Applicationscited by 59
- 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 42
- 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearningcited by 31
- 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Modelscited by 29
- 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 28
- 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detectioncited by 24
- 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 19
- 2025GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoningcited by 13
- 2026Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memorycited by 13
21 more
- 2025Evaluating System 1 vs. 2 Reasoning Approaches for Zero-Shot Time Series Forecasting: A Benchmark and Insightscited by 11
- 2025MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problemscited by 10
- 2025NeSyGeo: A Neuro-Symbolic Framework for Multimodal Geometric Reasoning Data Generationcited by 10
- 2025DrVD-Bench: Do Vision-Language Models Reason Like Human Doctors in Medical Image Diagnosis?cited by 10
- 2025PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensionscited by 8
- 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 8
- 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
- 2025EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Modelscited by 7
- 2025CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoringcited by 7
- 2025EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Abilitycited by 6
- 2025Knowing the Answer Isn't Enough: Fixing Reasoning Path Failures in LVLMscited by 5
- 2025DeepSound-V1: Start to Think Step-by-Step in the Audio Generation from Videoscited by 4
- 2025Reimagining Urban Science: Scaling Causal Inference with Large Language Modelscited by 3
- 2025Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysiscited by 3
- 2026LawThinker: A Deep Research Legal Agent in Dynamic Environmentscited by 3
- 2026MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Modelscited by 3
- 2025Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilitiescited by 2
- 2025Parameter Importance-Driven Continual Learning for Foundation Modelscited by 2
- 2025Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correctioncited by 1
- 2025Robust Diagram Reasoning: A Framework for Enhancing LVLM Performance on Visually Perturbed Scientific Diagramscited by 1
- 2026EAPO: Entropy-Driven Adaptive Positive-Negative Sample Weighting for Policy Optimization in Open-Ended QAcited by 0
- 2026MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Models24 Aug 2026
- 2026EAPO: Entropy-Driven Adaptive Positive-Negative Sample Weighting for Policy Optimization in Open-Ended QA27 May 2026
- 2026Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memory1 Apr 2026
- 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval23 Feb 2026
- 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework23 Feb 2026
- 2026LawThinker: A Deep Research Legal Agent in Dynamic Environments12 Feb 2026
- 2025Knowing the Answer Isn't Enough: Fixing Reasoning Path Failures in LVLMs6 Dec 2025
- 2025Parameter Importance-Driven Continual Learning for Foundation Models19 Nov 2025
- 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning14 Oct 2025
- 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruning28 Sep 2025
- 2025Implicit Reasoning in Large Language Models: A Comprehensive Survey2 Sep 2025
- 2025Robust Diagram Reasoning: A Framework for Enhancing LVLM Performance on Visually Perturbed Scientific Diagrams23 Aug 2025
21 more
- 2025GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning6 Aug 2025
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers30 Jun 2025
- 2025DrVD-Bench: Do Vision-Language Models Reason Like Human Doctors in Medical Image Diagnosis?30 May 2025
- 2025Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilities27 May 2025
- 2025NeSyGeo: A Neuro-Symbolic Framework for Multimodal Geometric Reasoning Data Generation21 May 2025
- 2025PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensions21 May 2025
- 2025Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis20 May 2025
- 2025CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring20 May 2025
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr 2025
- 2025Reimagining Urban Science: Scaling Causal Inference with Large Language Models15 Apr 2025
- 2025DeepSound-V1: Start to Think Step-by-Step in the Audio Generation from Videos28 Mar 2025
- 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection23 Mar 2025
- 2025LLM Agents for Education: Advances and Applications14 Mar 2025
- 2025EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Ability13 Mar 2025
- 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models12 Mar 2025
- 2025Evaluating System 1 vs. 2 Reasoning Approaches for Zero-Shot Time Series Forecasting: A Benchmark and Insights27 Feb 2025
- 2025MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems27 Feb 2025
- 2025Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correction21 Feb 2025
- 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning18 Feb 2025
- 2025EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models17 Feb 2025
- 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models16 Feb 2025
- 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 193
- 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
- 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 42
- 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 28
- 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detectioncited by 24
- 2025GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoningcited by 13
- 2025Evaluating System 1 vs. 2 Reasoning Approaches for Zero-Shot Time Series Forecasting: A Benchmark and Insightscited by 11
- 2025MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problemscited by 10
- 2025NeSyGeo: A Neuro-Symbolic Framework for Multimodal Geometric Reasoning Data Generationcited by 10
- 2025DrVD-Bench: Do Vision-Language Models Reason Like Human Doctors in Medical Image Diagnosis?cited by 10
- 2025PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensionscited by 8
12 most cited of 19 All 19 (JSON)
- 2026EAPO: Entropy-Driven Adaptive Positive-Negative Sample Weighting for Policy Optimization in Open-Ended QA27 May 2026
- 2026LawThinker: A Deep Research Legal Agent in Dynamic Environments12 Feb 2026
- 2025Knowing the Answer Isn't Enough: Fixing Reasoning Path Failures in LVLMs6 Dec 2025
- 2025Parameter Importance-Driven Continual Learning for Foundation Models19 Nov 2025
- 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoning14 Oct 2025
- 2025Implicit Reasoning in Large Language Models: A Comprehensive Survey2 Sep 2025
- 2025Robust Diagram Reasoning: A Framework for Enhancing LVLM Performance on Visually Perturbed Scientific Diagrams23 Aug 2025
- 2025GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning6 Aug 2025
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers30 Jun 2025
- 2025DrVD-Bench: Do Vision-Language Models Reason Like Human Doctors in Medical Image Diagnosis?30 May 2025
- 2025NeSyGeo: A Neuro-Symbolic Framework for Multimodal Geometric Reasoning Data Generation21 May 2025
- 2025PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensions21 May 2025
12 newest of 19 All 19 (JSON)
- 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearningcited by 31
- 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Modelscited by 29
- 2025EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Modelscited by 7
- 2026MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Modelscited by 3
- 2026MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Models24 Aug 2026
- 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning18 Feb 2025
- 2025EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models17 Feb 2025
- 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models16 Feb 2025
- 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 19
- 2026Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memorycited by 13
- 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 8
- 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
- 2025Reimagining Urban Science: Scaling Causal Inference with Large Language Modelscited by 3
- 2025Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysiscited by 3
- 2025Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correctioncited by 1
- 2026Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memory1 Apr 2026
- 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval23 Feb 2026
- 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework23 Feb 2026
- 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruning28 Sep 2025
- 2025Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis20 May 2025
- 2025Reimagining Urban Science: Scaling Causal Inference with Large Language Models15 Apr 2025
- 2025Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correction21 Feb 2025
Similar, not cited
Close in wording, no citation either way
- 2026SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation
- 2025Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoning
- 2025Scientific Reasoning: Assessment of Multimodal Generative LLMs
- 2025MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding
- 2025MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models
- 2024Can Multimodal Large Language Model Think Analogically?
- 2025Evaluating Large Language Models on Multimodal Chemistry Olympiad Exams
- 2025mmJEE-Eval: A Bilingual Multimodal Benchmark for Evaluating Scientific Reasoning in Vision-Language Models