# Event-Causal RAG: A Retrieval-Augmented Generation Framework for Long Video Reasoning in Complex Scenarios

arXiv: https://arxiv.org/abs/2605.06185 · HTML: https://mlsanity.com/paper/2605.06185/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 7 May 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Video understanding with MLLMs](https://mlsanity.com/mllm/video-understanding-with-mllms/)
- Retrieval-augmented generation / [Multimodal RAG](https://mlsanity.com/rag/multimodal-rag/)

## Builds on (most cited first, 18 of 18)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions](https://mlsanity.com/paper/2105.08276/) (2021)
- [LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models](https://mlsanity.com/paper/2407.07895/) (2024)
- [VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding](https://mlsanity.com/paper/2501.13106/) (2025)
- [HourVideo: 1-Hour Video-Language Understanding](https://mlsanity.com/paper/2411.04998/) (2024)
- [VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos](https://mlsanity.com/paper/2502.01549/) (2025)
- [From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understanding](https://mlsanity.com/paper/2409.18938/) (2024)
- [Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames](https://mlsanity.com/paper/2507.02001/) (2025)
- [Understanding Long Videos with Multimodal Language Models](https://mlsanity.com/paper/2403.16998/) (2024)
- [MECD: Unlocking Multi-Event Causal Discovery in Video Reasoning](https://mlsanity.com/paper/2409.17647/) (2024)
- [Causality Matters: How Temporal Information Emerges in Video Language Models](https://mlsanity.com/paper/2508.11576/) (2025)
- [Harnessing Temporal Causality for Advanced Temporal Action Detection](https://mlsanity.com/paper/2407.17792/) (2024)
- [CORM: Cache Optimization with Recent Message for Large Language Model Inference](https://mlsanity.com/paper/2404.15949/) (2024)
- [SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding](https://mlsanity.com/paper/2506.07600/) (2025)
- [HLV-1K: A Large-scale Hour-Long Video Benchmark for Time-Specific Long Video Understanding](https://mlsanity.com/paper/2501.01645/) (2025)
- [Visual Context Window Extension: A New Perspective for Long Video Understanding](https://mlsanity.com/paper/2409.20018/) (2024)
- [Temporal Contrastive Learning for Video Temporal Reasoning in Large Vision-Language Models](https://mlsanity.com/paper/2412.11391/) (2024)

## Abstract (arXiv)

Large vision-language models perform well on short- and medium-length video understanding but still struggle to maintain coherent event memory and recover long-range relationships in ultra-long videos. End-to-end methods are limited by visual-token growth and context length, while fixed-segment retrieval often fragments complete events and weakens state-transition modeling.We propose Event-Causal RAG (EC-RAG), a lightweight retrieval-augmented framework for ultra-long and streaming video reasoning. A dual visual-audio sentinel mechanism segments video streams into semantically complete events, represented as State-Event-State (SES) structures that organize observable pre-event states, central events, and post-event states as event-local causal transitions. These transitions are stored in dual vector-graph memory and temporally connected through entity-consistent trajectories. During question answering, bidirectional graph retrieval recovers relevant predecessor and successor events, and answers are generated using both structured memory and the corresponding video evidence.We further introduce ECV-1H, an hour-scale long-video QA benchmark dedicated to directed event-causal reasoning, with all source videos exceeding one hour. It covers over 150 hours of untrimmed video and contains 1,251 fully human-annotated QA pairs. EC-RAG improves overall accuracy by 4.96%--11.67% across three open-source video foundation models and achieves consistent gains across public datasets. On a single RTX 5090 GPU with 32 GB of memory, EC-RAG can continuously process videos while maintaining controlled streaming memory usage.