mlsanityData to 14 Oct 2026

MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems

arXiv 2503.01891 ↗First posted 27 Feb 2025cs.LG cs.CLcited by 10

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Recent advances in large language models (LLMs) and vision-language models (LVLMs) have shown promise across many tasks, yet their scientific reasoning capabilities remain untested, particularly in multimodal settings. We present MMSciBench, a benchmark for evaluating mathematical and physical reasoning through text-only and text-image formats, with human-annotated difficulty levels, solutions with detailed explanations, and taxonomic mappings. Evaluation of state-of-the-art models reveals significant limitations, with even the best model achieving only 63.77% accuracy and particularly struggling with visual reasoning tasks. Our analysis exposes critical gaps in complex reasoning and visual-textual integration, establishing MMSciBench as a rigorous standard for measuring progress in multimodal scientific understanding. The code for MMSciBench is open-sourced at GitHub, and the dataset is available at Hugging Face.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 41 references that are in the map. First posted on arXiv 27 Feb 2025.

Builds on

31 of 41 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning5 Feb
  2. 2024Zero-Shot Visual Reasoning by Vision-Language Models: Benchmarking and Analysis27 Aug
  3. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning10 Jan
  4. 2023Assessing GPT4-V on Structured Reasoning Tasks13 Dec

Other references · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  8. 2024A Survey on LLM-as-a-Judgecited by 1,423
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  10. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  11. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  12. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 372
15 more
  1. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  2. 2023Evaluating the Performance of Large Language Models on GAOKAO Benchmarkcited by 186
  3. 2021VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomenacited by 135
  4. 2023SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Researchcited by 115
  5. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 102
  6. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  7. 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 48
  8. 2024SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmarkcited by 35
  9. 2024Constructing Domain-Specific Evaluation Sets for LLM-as-a-judgecited by 30
  10. 2023GRASP: A novel benchmark for evaluating language GRounding And Situated Physics understanding in multimodal language modelscited by 28
  11. 2023TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Modelscited by 24
  12. 2023Monte Carlo Thought Search: Large Language Model Querying for Complex Scientific Reasoning in Catalyst Designcited by 17
  13. 2025DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Imagescited by 17
  14. 2023Benchmarking Zero-Shot Recognition with Vision-Language Models: Challenges on Granularity and Specificitycited by 14
  15. 2024GAOKAO-MM: A Chinese Human-Level Benchmark for Multimodal Models Evaluationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 4.

10 papers.

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  3. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  4. 2025Geoint-R1: Formalizing Multimodal Geometric Reasoning with Dynamic Auxiliary Constructionscited by 11
  5. 2026LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoningcited by 8
  6. 2025VisioMath: Benchmarking Figure-based Mathematical Reasoning in LMMscited by 6
  7. 2025GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Modelscited by 6
  8. 2026LiveK12Bench: Have Large Multimodal Models Truly Conquered High School-level Examinations?cited by 3
  9. 2026How RL Unlocks the Aha Moment in Geometric Interleaved Reasoningcited by 2
  10. 2026Science Edge Evaluation: SEE the Missing Step Toward Real Scientific Discoverycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning Capabilities of VLMs
  2. 2025MAC: A Live Benchmark for Multimodal Large Language Models in Scientific Understanding
  3. 2025MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Models
  4. 2025mmJEE-Eval: A Bilingual Multimodal Benchmark for Evaluating Scientific Reasoning in Vision-Language Models
  5. 2024CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmark
  6. 2025Multi-Physics: A Comprehensive Benchmark for Multimodal LLMs Reasoning on Chinese Multi-Subject Physics Problems
  7. 2026SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation
  8. 2026WorldBench: A Challenging and Visually Diverse Multimodal Reasoning Benchmark