# Video-CoE: Reinforcing Video Event Prediction via Chain of Events

arXiv: https://arxiv.org/abs/2603.14935 · HTML: https://mlsanity.com/paper/2603.14935/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 60 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Builds on (most cited first, 30 of 49)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation](https://mlsanity.com/paper/2307.06942/) (2023)
- [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025)
- [GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://mlsanity.com/paper/2507.01006/) (2025)
- [Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model](https://mlsanity.com/paper/2503.24290/) (2025)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [TempCompass: Do Video LLMs Really Understand Videos?](https://mlsanity.com/paper/2403.00476/) (2024)
- [VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos](https://mlsanity.com/paper/2405.19209/) (2024)
- [Token-Budget-Aware LLM Reasoning](https://mlsanity.com/paper/2412.18547/) (2024)
- [Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos](https://mlsanity.com/paper/2501.13826/) (2025)
- [Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding](https://mlsanity.com/paper/2409.14485/) (2024)
- [Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs](https://mlsanity.com/paper/2407.21771/) (2024)
- [GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning](https://mlsanity.com/paper/2504.02546/) (2025)
- [MMVU: Measuring Expert-Level Multi-Discipline Video Understanding](https://mlsanity.com/paper/2501.12380/) (2025)
- [See What You Are Told: Visual Attention Sink in Large Multimodal Models](https://mlsanity.com/paper/2503.03321/) (2025)
- [GRIT: Teaching MLLMs to Think with Images](https://mlsanity.com/paper/2505.15879/) (2025)
- [IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding](https://mlsanity.com/paper/2402.18476/) (2024)
- [VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning](https://mlsanity.com/paper/2505.12434/) (2025)
- [Apollo: An Exploration of Video Understanding in Large Multimodal Models](https://mlsanity.com/paper/2412.10360/) (2024)
- [UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning](https://mlsanity.com/paper/2505.14231/) (2025)
- [VideoCLIP-XL: Advancing Long Description Understanding for Video CLIP Models](https://mlsanity.com/paper/2410.00741/) (2024)

## Built on by (most cited first, 4 of 4)

- [Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction](https://mlsanity.com/paper/2606.05769/) (2026)
- [IndusAgent: Reinforcing Open-Vocabulary Industrial Anomaly Detection with Agentic Tools](https://mlsanity.com/paper/2605.20682/) (2026)
- [From Sight to Foresight: Predictive Spatial Reasoning in Vision-Language Models](https://mlsanity.com/paper/2610.04139/) (2026)
- [VepAgent: Bridging Causal-Transition via Tool-Augmented Reinforcement Learning for Video Event Prediction](https://mlsanity.com/paper/2610.06293/) (2026)

## Abstract (arXiv)

Despite advances in the application of MLLMs for various video tasks, video event prediction (VEP) remains relatively underexplored. VEP requires the model to perform fine-grained temporal modeling of videos and establish logical relationships between videos and future events, which current MLLMs still struggle with. In this work, we first present a comprehensive evaluation of current leading MLLMs on the VEP task, revealing the reasons behind their inaccurate predictions, including lack of logical reasoning ability for future events prediction and insufficient utilization of visual information. To address these challenges, we propose Chain of Events (CoE) paradigm, which constructs temporal event chains to implicitly enforce MLLM focusing on the visual content and the logical connections between videos and future events, incentivizing model's reasoning capability with multiple training protocols. Experimental results on public benchmarks demonstrate that our method outperforms both leading open-source and commercial MLLMs, establishing a new state-of-the-art on the VEP task. Codes and models will be released soon.