GCAgent: Long-Video Understanding via Schematic and Narrative Episodic Memory
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
Abstract (arXiv)
Long-video understanding remains a significant challenge for Multimodal Large Language Models (MLLMs) due to inherent token limitations and the complexity of capturing long-term temporal dependencies. Existing methods often fail to capture the global context and complex event relationships necessary for deep video reasoning. To address this, we introduce GCAgent, a novel Global-Context-Aware Agent framework that achieves comprehensive long-video understanding. Our core innovation is the Schematic and Narrative Episodic Memory. This memory structurally models events and their causal and temporal relations into a concise, organized context, fundamentally resolving the long-term dependency problem. Operating in a multi-stage Perception-Action-Reflection cycle, our GCAgent utilizes a Memory Manager to retrieve relevant episodic context for robust, context-aware inference. Extensive experiments confirm that GCAgent significantly enhances long-video understanding, achieving up to 23.5% accuracy improvement on the Video-MME Long split over a strong MLLM baseline. Furthermore, our framework establishes state-of-the-art performance among comparable 7B-scale MLLMs, achieving 73.4% accuracy on the Long split and the highest overall average (71.9%) on the Video-MME benchmark, validating our agent-based reasoning paradigm and structured memory for cognitively-inspired long-video understanding.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 55 of its 63 references that are in the map. First posted on arXiv 15 Nov 2025.
Builds on
55 of 63 references.
Closest ancestors · 17
Same direction, newest first
- 2024SALOVA: Segment-Augmented Long Video Assistant for Targeted Retrieval and Routing in Long-Form Video Analysis25 Nov
- 2024TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning25 Oct
- 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding22 Oct
- 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
- 2024LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecture4 Sep
- 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videos26 Aug
- 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videos19 Aug
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
- 2024Long Context Transfer from Language to Vision24 Jun
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
- 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning25 Apr
- 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding8 Apr
5 more
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding18 Mar
- 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmark28 Nov
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun
Other references · 38
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023Qwen Technical Reportcited by 3,344
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
- 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
26 more
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
- 2019VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Researchcited by 414
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
- 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
- 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
- 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 214
- 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 170
- 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 159
- 2022FlexiViT: One Model for All Patch Sizescited by 152
- 2024Direct Preference Optimization of Video Large Multimodal Models from Language Model Rewardcited by 147
- 2024LongVLM: Efficient Long Video Understanding via Large Language Modelscited by 147
- 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 138
- 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 137
- 2024DrVideo: Document Retrieval Based Long Video Understandingcited by 65
- 2025LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agentscited by 58
- 2025VideoRAG: Retrieval-Augmented Generation over Video Corpuscited by 57
- 2024VCA: Video Curious Agent for Long Video Understandingcited by 56
- 2024KeyVideoLLM: Towards Large-scale Video Keyframe Selectioncited by 54
- 2022Hierarchical Local-Global Transformer for Temporal Sentence Groundingcited by 38
- 2024LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaborationcited by 38
- 2025SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understandingcited by 30
- 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 25
- 2024SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet Knowledgecited by 20
- 2022Locate before Answering: Answer Guided Question Localization for Video Question Answeringcited by 19
- 2025Multimodal Large Models Are Effective Action Anticipatorscited by 10
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 44
- 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 4
- 2026REVEAL: A Rubric-Guided Agent for Explicit Evidence Sufficiency Verificationin Long-Video Question Answeringcited by 1
- 2026video-SALMONN-R3: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understandingcited by 0
- 2026REVEAL: A Rubric-Guided Agent for Explicit Evidence Sufficiency Verificationin Long-Video Question Answering9 Aug 2026
- 2026video-SALMONN-R3: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understanding23 Jun 2026
- 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMs5 Jun 2026
- 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents14 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2026PyraVid: Hierarchical Multimodal Memory for Long-Horizon Video Reasoning
- 2026Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning
- 2026Visual Agentic Memory: Enabling Online Long Video Understanding via Online Indexing, Hierarchical Memory, and Agentic Retrieval
- 2026VideoTapestry: Query-Adaptive Memory Refinement for Multi-Agent Long-Video Understanding
- 2025WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning
- 2026LongVideo-R1: Smart Navigation for Low-cost Long Video Understanding
- 2026VideoScout: Learning Agentic Active Exploration with Adaptive Reasoning Pacing for Long Video Understanding
- 2026From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents