# OMIBench: Benchmarking Olympiad-Level Multi-Image Reasoning in Large Vision-Language Model arXiv: https://arxiv.org/abs/2604.20806 ยท HTML: https://mlsanity.com/paper/2604.20806/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. - Builds on 33 of its 78 references that are in the map. - First posted on arXiv 22 Apr 2026. ## Placed in - LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/) - Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/) ## Builds on (most cited first, 30 of 33) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025) - [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021) - [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024) - [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025) - [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025) - [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025) - [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025) - [Kimi-VL Technical Report](https://mlsanity.com/paper/2504.07491/) (2025) - [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025) - [Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning](https://mlsanity.com/paper/2105.04165/) (2021) - [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025) - [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025) - [Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2503.12605/) (2025) - [UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression](https://mlsanity.com/paper/2212.02746/) (2022) - [MiMo-VL Technical Report](https://mlsanity.com/paper/2506.03569/) (2025) - [Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Models](https://mlsanity.com/paper/2503.21380/) (2025) - [MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models](https://mlsanity.com/paper/2408.02718/) (2024) - [AI4Research: A Survey of Artificial Intelligence for Scientific Research](https://mlsanity.com/paper/2507.01903/) (2025) - [SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning](https://mlsanity.com/paper/2506.01713/) (2025) - [Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning](https://mlsanity.com/paper/2406.12742/) (2024) - [Kwai Keye-VL Technical Report](https://mlsanity.com/paper/2507.01949/) (2025) - [Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm](https://mlsanity.com/paper/2511.04570/) (2025) - [MIBench: Evaluating Multimodal Large Language Models over Multiple Images](https://mlsanity.com/paper/2407.15272/) (2024) - [Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought](https://mlsanity.com/paper/2505.15510/) (2025) - [Slow Perception: Let's Perceive Geometric Figures Step-by-step](https://mlsanity.com/paper/2412.20631/) (2024) - [Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmark](https://mlsanity.com/paper/2506.04280/) (2025) - [HiPhO: How Far Are (M)LLMs from Humans in the Latest High School Physics Olympiad Benchmark?](https://mlsanity.com/paper/2509.07894/) (2025) - [MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI](https://mlsanity.com/paper/2506.23563/) (2025) - [Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation](https://mlsanity.com/paper/2508.12680/) (2025) ## Built on by (most cited first, 5 of 5) - [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026) - [Rethinking Critic Learning in PPO: Understanding and Mitigating Value Flattening](https://mlsanity.com/paper/2609.18708/) (2026) - [StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning](https://mlsanity.com/paper/2606.00148/) (2026) - [SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World Repositories](https://mlsanity.com/paper/2609.29465/) (2026) - [MVGraph: Benchmarking LVLMs on Evidence Composition across Multi-View Visual Graph](https://mlsanity.com/paper/2610.12961/) (2026) ## Abstract (arXiv) Large vision-language models (LVLMs) have made substantial advances in reasoning tasks at the Olympiad level. Nevertheless, current Olympiad-level multimodal reasoning benchmarks for these models often emphasize single-image analysis and fail to exploit contextual information across multiple images. We present OMIBench, a benchmark designed to evaluate Olympiad-level reasoning when the required evidence is distributed over multiple images. It contains problems from biology, chemistry, mathematics, and physics Olympiads, together with manually annotated rationales and evaluation protocols for both exact and semantic answer matching. Across extensive experiments on OMIBench, we observe meaningful performance gaps in existing models. Even the strongest LVLMs, such as Gemini-3-Pro, attain only about 50% on the benchmark. These results position OMIBench as a focused resources for studying and improving multi-image reasoning in LVLMs.