mlsanityData to 14 Oct 2026

R-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation

arXiv 2505.02018 ↗First posted 4 May 2025cs.CVcited by 34

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Reasoning stands as a cornerstone of intelligence, enabling the synthesis of existing knowledge to solve complex problems. Despite remarkable progress, existing reasoning benchmarks often fail to rigorously evaluate the nuanced reasoning capabilities required for complex, real-world problemsolving, particularly in multi-disciplinary and multimodal contexts. In this paper, we introduce a graduate-level, multi-disciplinary, EnglishChinese benchmark, dubbed as Reasoning Bench (R-Bench), for assessing the reasoning capability of both language and multimodal models. RBench spans 1,094 questions across 108 subjects for language model evaluation and 665 questions across 83 subjects for multimodal model testing in both English and Chinese. These questions are meticulously curated to ensure rigorous difficulty calibration, subject balance, and crosslinguistic alignment, enabling the assessment to be an Olympiad-level multi-disciplinary benchmark. We evaluate widely used models, including OpenAI o1, GPT-4o, DeepSeek-R1, etc. Experimental results indicate that advanced models perform poorly on complex reasoning, especially multimodal reasoning. Even the top-performing model OpenAI o1 achieves only 53.2% accuracy on our multimodal evaluation. Data and code are made publicly available at here.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 63 references that are in the map. First posted on arXiv 4 May 2025.

Builds on

36 of 63 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  2. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  3. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct

Other references · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  10. 2023Mistral 7Bcited by 3,164
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  12. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
21 more
  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  4. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  5. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  6. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  7. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  8. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  9. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  10. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  11. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  12. 2022A Survey on In-context Learningcited by 721
  13. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  14. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  15. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  16. 2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 284
  17. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
  18. 2024Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positivecited by 241
  19. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 218
  20. 2024Are Your LLMs Capable of Stable Reasoning?cited by 65
  21. 2024Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMscited by 40

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 15; 2026: 19.

34 papers.

Sort:

By area:

  1. 2025When AI Co-Scientists Fail: SPOT-a Benchmark for Automated Verification of Scientific Researchcited by 26
  2. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  3. 2025ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Mapscited by 23
  4. 2025GenExam: A Multidisciplinary Text-to-Image Examcited by 22
  5. 2025RBench-V: A Primary Assessment for Visual Reasoning Models with Multi-modal Outputscited by 17
  6. 2025NVIDIA Nemotron Nano V2 VLcited by 17
  7. 2026SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agentscited by 15
  8. 2025AgentFrontier: Expanding the Capability Frontier of LLM Agents with ZPD-Guided Data Synthesiscited by 14
  9. 2025OPT-BENCH: Evaluating LLM Agent on Large-Scale Search Spaces Optimization Problemscited by 12
  10. 2025BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Datasetcited by 12
  11. 2025SciAgent: A Unified Multi-Agent System for Generalistic Scientific Reasoningcited by 7
  12. 2025MME-SCI: A Comprehensive and Challenging Science Benchmark for Multimodal Large Language Modelscited by 6
22 more
  1. 2026Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budgetcited by 6
  2. 2025NOSA: Native and Offloadable Sparse Attentioncited by 4
  3. 2026TangramPuzzle: Evaluating Multimodal Large Language Models with Compositional Spatial Reasoningcited by 4
  4. 2026Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoningcited by 4
  5. 2026IRIS: A Real-World Benchmark for Inverse Recovery and Identification of Physical Dynamic Systems from Monocular Videocited by 4
  6. 2025Verifying Meta-Awareness via Predictive Rewards in Reasoning Modelscited by 3
  7. 2026InfiCoEvalChain: A Blockchain-Based Decentralized Framework for Collaborative LLM Evaluationcited by 3
  8. 2026MapTab: A Diagnostic Benchmark for Long-Horizon Multi-Criteria Multimodal Reasoning on Heterogeneous Topological Graphscited by 3
  9. 2026GeoTikzBridge: Advancing Multimodal Code Generation for Geometric Perception and Reasoningcited by 3
  10. 2026BotzoneBench: Scalable LLM Evaluation via Graded AI Anchorscited by 2
  11. 2026OmniMapBench: Benchmarking Visual-Centric Reasoning on Diverse Map Documentscited by 2
  12. 2026AEGIS: Exploring the Limit of World Knowledge Capabilities for Unified Mulitmodal Modelscited by 1
  13. 2025Aligning MLLM Benchmark With Human Preferences via Structural Equation Modelingcited by 0
  14. 2025Prompt2Craft: Generating Functional Craft Assemblies with LLMscited by 0
  15. 2026ChronoPhyBench: Do MLLMs Truly Understand the World or Merely Exploit Language Priors?cited by 0
  16. 2026Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extractioncited by 0
  17. 2026SFBench: The SciFy Scientific Feasibility Benchmarkcited by 0
  18. 2026Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Modelscited by 0
  19. 2026Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoningcited by 0
  20. 2026LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillationcited by 0
  21. 2026Visual-Invariance-Augmented Feature Optimal Alignment for Transferable Adversarial Attacks against Closed-Source MLLMscited by 0
  22. 2026Degrees of Freedom in Multi-Agent Coordination: Between Centralized and Decentralized Regimescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models
  2. 2025MDK12-Bench: A Comprehensive Evaluation of Multimodal Large Language Models on Multidisciplinary Exams
  3. 2025Human-Aligned Bench: Fine-Grained Assessment of Reasoning Ability in MLLMs vs. Humans
  4. 2025The Ouroboros of Benchmarking: Reasoning Evaluation in an Era of Saturation
  5. 2023InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models
  6. 2025ReasoningWeekly: A General Knowledge and Verbal Reasoning Challenge for Large Language Models
  7. 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Models
  8. 2025RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation