EventLens: Event-Structure Reinforcement Learning for Video Understanding
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
Abstract (arXiv)
Recent multimodal large language models (MLLMs) achieve strong results on video question answering, yet often fail to recover the temporal structure that makes a video coherent. They may merge neighboring events, miss single-event progression, or infer implausible relations across events. We study structured temporal understanding as the ability to recover event structure from visual evidence, including event boundaries, single-event progression, and multi-event relations. We argue that this gap arises from a mismatch between current post-training objectives and temporal reasoning, where caption-style supervision allows models to rely on language shortcuts rather than visual temporal grounding. To address this, we propose EventLens, a vision-centric reinforcement learning framework that learns video understanding through event-structure recovery. Instead of reconstructing captions, EventLens trains models to recover temporal structure under controlled perturbations, such as altered temporal granularity, reversed progression, and shuffled event order. We instantiate this framework with a three-level temporal ontology and derive three verifiable task families: event segmentation, progression discrimination, and multi-event relation reasoning. These tasks admit deterministic rewards, enabling task-specialized GRPO training without human preference labels or learned reward models. We further introduce multi-teacher on-policy distillation (MT-OPD) to consolidate specialized policies into a unified model. Experiments on Qwen3-VL backbones show consistent improvements on temporal grounding and event-sensitive reasoning benchmarks, while preserving general video-QA performance. Learning curves further demonstrate that EventLens achieves comparable downstream transfer with significantly reduced training cost compared to direct mixed RL optimization.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 53 of its 57 references that are in the map. First posted on arXiv 12 Oct 2026.
Builds on
53 of 57 references.
Closest ancestors · 22
Same direction, newest first
- 2026MVPBench: A Multi-Video Perception Evaluation Benchmark for Multi-Modal Video Understanding24 Mar
- 2026Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillation3 Feb
- 2026VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twice8 Jan
- 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs16 Dec
- 2025What Happens When: Learning Temporal Orders of Events in Videos5 Dec
- 2025TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learning3 Dec
- 2025OneThinker: All-in-one Reasoning Model for Image and Video2 Dec
- 2025ViSS-R1: Self-Supervised Reinforcement Video Reasoning17 Nov
- 2025VideoSSR: Video Self-Supervised Reinforcement Learning9 Nov
- 2025TempCore: Are Video QA Benchmarks Temporally Grounded?1 Sep
- 2025VidBridge-R1: Bridging QA and Captioning for RL-based Video Understanding Models with Intermediate Proxy Tasks10 Jun
- 2025Seeing the Arrow of Time in Large Multimodal Models3 Jun
10 more
- 2025MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding27 May
- 2025TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action2 May
- 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
- 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding17 Mar
- 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling21 Jan
- 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning25 Oct
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
- 2024Towards Event-oriented Long Video Understanding20 Jun
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmark28 Nov
Other references · 31
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
- 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 373
- 2024MLVU: Benchmarking Multi-task Long Video Understandingcited by 292
- 2017Unsupervised Representation Learning by Sorting Sequencescited by 221
19 more
- 2021ASFormer: Transformer for Action Segmentationcited by 168
- 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 118
- 2020Alleviating Over-segmentation Errors by Detecting Action Boundariescited by 87
- 2020Video Representation Learning by Recognizing Temporal Transformationscited by 83
- 2020Video Cloze Procedure for Self-Supervised Spatio-Temporal Learningcited by 81
- 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 70
- 2021Generic Event Boundary Detection: A Benchmark for Event Segmentationcited by 65
- 2024Lost in Time: A New Temporal Benchmark for VideoLLMscited by 42
- 2021One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teacherscited by 40
- 2025Visual Jigsaw Post-Training Improves MLLMscited by 26
- 2021Progressive Attention on Multi-Level Dense Difference Maps for Generic Event Boundary Detectioncited by 20
- 2025Breaking Down Video LLM Benchmarks: Knowledge, Spatial Perception, or True Temporal Understanding?cited by 20
- 2025Fostering Video Reasoning via Next-Event Predictioncited by 18
- 2024Reversed in Time: A Novel Temporal-Emphasized Benchmark for Cross-Modal Video-Text Retrievalcited by 13
- 2025Commonsense Video Question Answering through Video-Grounded Entailment Tree Reasoningcited by 12
- 2025VideoExpert: Augmented LLM for Temporal-Sensitive Video Understandingcited by 9
- 2025R1-Track: Direct Application of MLLMs to Visual Object Tracking via Reinforcement Learningcited by 9
- 2025Online Generic Event Boundary Detectioncited by 5
- 2026ViewSAM: Learning View-aware Cross-modal Semantics for Weakly Supervised Cross-view Referring Multi-Object Trackingcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinking
- 2026TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs
- 2026ViKey: Enhancing Temporal Understanding in Videos via Visual Prompting
- 2026Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
- 2026Before It Fades: Reinforcing Temporal Representations at Inference Time in VideoLLMs
- 2025Causality Matters: How Temporal Information Emerges in Video Language Models
- 2026Training-Free Temporal Abstraction for General Video Understanding
- 2026EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Use