EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Use
Not on the map. Why?
Abstract (arXiv)
Online video understanding requires models to perform continuous perception and long-range reasoning within potentially infinite visual streams. Its fundamental challenge lies in the conflict between the unbounded nature of streaming media input and the limited context window of Multimodal Large Language Models (MLLMs). Current methods primarily rely on passive processing, which often face a trade-off between maintaining long-range context and capturing the fine-grained details necessary for complex tasks. To address this, we introduce EventMemAgent, an active online video agent framework based on a hierarchical memory module. Our framework employs a dual-layer strategy for online videos: short-term memory detects event boundaries and utilizes event-granular reservoir sampling to process streaming video frames within a fixed-length buffer dynamically; long-term memory structuredly archives past observations on an event-by-event basis. Furthermore, we integrate a multi-granular perception toolkit for active, iterative evidence capture and employ Agentic Reinforcement Learning (Agentic RL) to end-to-end internalize reasoning and tool-use strategies into the agent's intrinsic capabilities. Experiments show that EventMemAgent achieves competitive results on online video benchmarks. The code will be released here: https://github.com/lingcco/EventMemAgent.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 47 of its 52 references that are in the map. First posted on arXiv 17 Feb 2026.
Builds on
47 of 52 references.
References · 47
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Visual Instruction Tuningcited by 7,792
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
35 more
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
- 2024Long Context Transfer from Language to Visioncited by 502
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 323
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
- 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
- 2025DeepSeek-OCR: Contexts Optical Compressioncited by 174
- 2024Streaming Long Video Understanding with Large Language Modelscited by 164
- 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 160
- 2024StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understandingcited by 135
- 2025OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?cited by 134
- 2024Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Inputcited by 132
- 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
- 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 112
- 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videoscited by 95
- 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 90
- 2025StreamingVLM: Real-Time Understanding for Infinite Video Streamscited by 89
- 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 88
- 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
- 2025StreamForest: Efficient Online Video Understanding with Persistent Event Memorycited by 71
- 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
- 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 63
- 2025TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoningcited by 59
- 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 58
- 2025VideoAgent2: Enhancing the LLM-Based Agent System for Long-Form Video Understanding by Uncertainty-Aware CoTcited by 36
- 2026HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understandingcited by 36
- 2025StreamAgent: Towards Anticipatory Agents for Streaming Video Understandingcited by 29
- 2025StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compressioncited by 24
- 2025Unleashing Hour-Scale Video Training for Long Video-Language Understandingcited by 21
- 2025VideoLucy: Deep Memory Backtracking for Long Video Understandingcited by 21
- 2025Venus: An Efficient Edge Memory-and-Retrieval System for VLM-based Online Video Understandingcited by 4
- 2025TinyLLaVA-Video: Towards Smaller LMMs for Video Understanding with Group Resamplercited by 2
- 2025Perceive, Verify and Understand Long Video: Multi-Granular Perception and Active Verification via Interactive Agentscited by 2
- 2025VideoScaffold: Elastic-Scale Visual Hierarchies for Streaming Video Understanding in MLLMscited by 1
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneouslycited by 19
- 2026FileGram: Grounding Agent Personalization in File-System Behavioral Tracescited by 3
- 2026Bridging Modalities, Spanning Time: Structured Memory for Ultra-Long Agentic Video Reasoningcited by 3
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understandingcited by 3
- 2026Visual Agentic Memory: Enabling Online Long Video Understanding via Online Indexing, Hierarchical Memory, and Agentic Retrievalcited by 2
- 2026ShallowStream: Index Shallow then Answer Deep for Streaming Video Understandingcited by 2
- 2026Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understandingcited by 2
- 2026SCOUT: Self-Checking and Recovery-Aware Tool-Thought Agents for Ultra-Long Egocentric Video Reasoningcited by 0
- 2026EM^2Mem: Event-Centric Multimodal Memory for Large Language Modelscited by 0
- 2026FlashBack: Knowing When to Remember in Streaming Vision-Language Modelscited by 0
- 2026FlashBack: Knowing When to Remember in Streaming Vision-Language Models1 Oct 2026
- 2026Caption-once, Frames-on-Demand: Visual-Need Routing for Budget-Aware Agentic Long Video Understanding10 Sep 2026
- 2026ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding2 Sep 2026
- 2026EM^2Mem: Event-Centric Multimodal Memory for Large Language Models1 Sep 2026
- 2026SCOUT: Self-Checking and Recovery-Aware Tool-Thought Agents for Ultra-Long Egocentric Video Reasoning8 Aug 2026
- 2026FOLIO: Focused Semantic Memory for Streaming Video Understanding14 Jul 2026
- 2026Visual Agentic Memory: Enabling Online Long Video Understanding via Online Indexing, Hierarchical Memory, and Agentic Retrieval15 May 2026
- 2026Bridging Modalities, Spanning Time: Structured Memory for Ultra-Long Agentic Video Reasoning8 May 2026
- 2026FileGram: Grounding Agent Personalization in File-System Behavioral Traces6 Apr 2026
- 2026Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously12 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Agentic Video Understanding: A Survey
- 2026VideoTapestry: Query-Adaptive Memory Refinement for Multi-Agent Long-Video Understanding
- 2026MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks
- 2026Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding
- 2026VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understanding
- 2025VideoARM: Agentic Reasoning over Hierarchical Memory for Long-Form Video Understanding
- 2024Adaptive Video Understanding Agent: Enhancing efficiency with dynamic frame sampling and feedback-driven reasoning