# VRBench: A Benchmark for Multi-Step Reasoning in Long Narrative Videos

arXiv: https://arxiv.org/abs/2506.10857 · HTML: https://mlsanity.com/paper/2506.10857/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 55 of its 94 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 30 of 55)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding](https://mlsanity.com/paper/2501.13106/) (2025)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark](https://mlsanity.com/paper/2409.02813/) (2024)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [LLaVA-CoT: Let Vision Language Models Reason Step-by-Step](https://mlsanity.com/paper/2411.10440/) (2024)
- [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024)
- [InternLM2 Technical Report](https://mlsanity.com/paper/2403.17297/) (2024)
- [Kimi-VL Technical Report](https://mlsanity.com/paper/2504.07491/) (2025)
- [TempCompass: Do Video LLMs Really Understand Videos?](https://mlsanity.com/paper/2403.00476/) (2024)
- [MLVU: Benchmarking Multi-task Long Video Understanding](https://mlsanity.com/paper/2406.04264/) (2024)
- [VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2504.06958/) (2025)
- [Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization](https://mlsanity.com/paper/2411.10442/) (2024)
- [Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos](https://mlsanity.com/paper/2501.13826/) (2025)
- [VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling](https://mlsanity.com/paper/2501.00574/) (2024)

## Built on by (most cited first, 24 of 24)

- [Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning](https://mlsanity.com/paper/2510.23473/) (2025)
- [Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidence](https://mlsanity.com/paper/2510.20470/) (2025)
- [Watching, Reasoning, and Searching: A Video Deep Research Benchmark on Open Web for Agentic Video Reasoning](https://mlsanity.com/paper/2601.06943/) (2026)
- [JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligence](https://mlsanity.com/paper/2606.14777/) (2026)
- [MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn Dialogues](https://mlsanity.com/paper/2510.17722/) (2025)
- [ExpVid: A Benchmark for Experiment Video Understanding & Reasoning](https://mlsanity.com/paper/2510.11606/) (2025)
- [VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs](https://mlsanity.com/paper/2511.20272/) (2025)
- [Watch, Remember, Reason: Human-View Video Understanding with MLLMs](https://mlsanity.com/paper/2606.07433/) (2026)
- [NeMo: Needle in a Montage for Video-Language Understanding](https://mlsanity.com/paper/2509.24563/) (2025)
- [Benchmarking PhD-Level Coding in 3D Geometric Computer Vision](https://mlsanity.com/paper/2603.30038/) (2026)
- [TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding](https://mlsanity.com/paper/2608.22516/) (2026)
- [MINERVA-Cultural: A Benchmark for Cultural and Multilingual Long Video Reasoning](https://mlsanity.com/paper/2601.10649/) (2026)
- [Chain-of-Glimpse: Search-Guided Progressive Object-Grounded Reasoning for Video Understanding](https://mlsanity.com/paper/2604.14692/) (2026)
- [Towards Temporal Compositional Reasoning in Long-Form Sports Videos](https://mlsanity.com/paper/2604.22226/) (2026)
- [SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Series](https://mlsanity.com/paper/2606.03301/) (2026)
- [Hierarchical Denoising For Multi-Step Visual Reasoning](https://mlsanity.com/paper/2607.15278/) (2026)
- [Narrative Aligned Long Form Video Question Answering](https://mlsanity.com/paper/2603.19481/) (2026)
- [CaST-Bench: Benchmarking Causal Chain-Grounded Spatio-Temporal Reasoning for Video Question Answering](https://mlsanity.com/paper/2605.23216/) (2026)
- [MetaphorVU: Towards Metaphorical Video Understanding](https://mlsanity.com/paper/2605.25461/) (2026)
- [StoryVideoQA: Scaling Deep Video Understanding with a Large-Scale, Multi-Genre and Auto-Generated Dataset](https://mlsanity.com/paper/2606.06338/) (2026)
- [VideoLatent: Video-Language Learning via Latent Self-Forcing](https://mlsanity.com/paper/2606.22870/) (2026)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)
- [DVBench: Benchmarking MLLMs for Understanding Dynamic Charts and Narratives in Data Videos](https://mlsanity.com/paper/2608.29711/) (2026)
- [Video-HopChain: Multi-Hop Questions and Confidence-Gated Exploration for Video Reasoning Models](https://mlsanity.com/paper/2609.25773/) (2026)

## Abstract (arXiv)

We present VRBench, the first long narrative video benchmark crafted for evaluating large models' multi-step reasoning capabilities, addressing limitations in existing evaluations that overlook temporal reasoning and procedural validity. It comprises 960 long videos (with an average duration of 1.6 hours), along with 8,243 human-labeled multi-step question-answering pairs and 25,106 reasoning steps with timestamps. These videos are curated via a multi-stage filtering process including expert inter-rater reviewing to prioritize plot coherence. We develop a human-AI collaborative framework that generates coherent reasoning chains, each requiring multiple temporally grounded steps, spanning seven types (e.g., event attribution, implicit inference). VRBench designs a multi-phase evaluation pipeline that assesses models at both the outcome and process levels. Apart from the MCQs for the final results, we propose a progress-level LLM-guided scoring metric to evaluate the quality of the reasoning chain from multiple dimensions comprehensively. Through extensive evaluations of 12 LLMs and 19 VLMs on VRBench, we undertake a thorough analysis and provide valuable insights that advance the field of multi-step reasoning.