mlsanityData to 14 Oct 2026

OMIBench: Benchmarking Olympiad-Level Multi-Image Reasoning in Large Vision-Language Model

arXiv 2604.20806 ↗First posted 22 Apr 2026cs.CV cs.AI cs.CLcited by 5

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Large vision-language models (LVLMs) have made substantial advances in reasoning tasks at the Olympiad level. Nevertheless, current Olympiad-level multimodal reasoning benchmarks for these models often emphasize single-image analysis and fail to exploit contextual information across multiple images. We present OMIBench, a benchmark designed to evaluate Olympiad-level reasoning when the required evidence is distributed over multiple images. It contains problems from biology, chemistry, mathematics, and physics Olympiads, together with manually annotated rationales and evaluation protocols for both exact and semantic answer matching. Across extensive experiments on OMIBench, we observe meaningful performance gaps in existing models. Even the strongest LVLMs, such as Gemini-3-Pro, attain only about 50% on the benchmark. These results position OMIBench as a focused resources for studying and improving multi-image reasoning in LVLMs.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 78 references that are in the map. First posted on arXiv 22 Apr 2026.

Builds on

33 of 78 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025ChainV: Atomic Visual Hints Make Multimodal Reasoning Shorter and Better21 Nov
  2. 2025Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm6 Nov
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug
  4. 2025Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation18 Aug
  5. 2025Semi-off-Policy Reinforcement Learning for Vision-Language Slow-Thinking Reasoning22 Jul
  6. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers30 Jun
  7. 2025MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI30 Jun
  8. 2025Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark4 Jun
  9. 2025MiMo-VL Technical Report4 Jun
  10. 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning2 Jun
  11. 2025Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought21 May
  12. 2025Kimi-VL Technical Report10 Apr
2 more
  1. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey16 Mar
  2. 2024Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning18 Jun

Other references · 19

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  6. 2025Qwen3-VL Technical Reportcited by 2,793
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  8. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  9. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 383
  10. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  11. 2022UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expressioncited by 137
  12. 2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Modelscited by 70
7 more
  1. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 67
  2. 2025AI4Research: A Survey of Artificial Intelligence for Scientific Researchcited by 60
  3. 2025Kwai Keye-VL Technical Reportcited by 47
  4. 2024MIBench: Evaluating Multimodal Large Language Models over Multiple Imagescited by 39
  5. 2024Slow Perception: Let's Perceive Geometric Figures Step-by-stepcited by 24
  6. 2025HiPhO: How Far Are (M)LLMs from Humans in the Latest High School Physics Olympiad Benchmark?cited by 19
  7. 2026The Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  2. 2026Rethinking Critic Learning in PPO: Understanding and Mitigating Value Flatteningcited by 2
  3. 2026StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoningcited by 0
  4. 2026SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World Repositoriescited by 0
  5. 2026MVGraph: Benchmarking LVLMs on Evidence Composition across Multi-View Visual Graphcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling
  2. 2025Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency
  3. 2025Visual Reasoning Evaluation of Grok, Deepseek Janus, Gemini, Qwen, Mistral, and ChatGPT
  4. 2024NPHardEval4V: Dynamic Evaluation of Large Vision-Language Models with Effects of Vision
  5. 2024A Cognitive Evaluation Benchmark of Image Reasoning and Description for Large Vision-Language Models
  6. 2026MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoning
  7. 2025VisChainBench: A Benchmark for Multi-Turn, Multi-Image Visual Reasoning Beyond Language Priors
  8. 2026Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints