# Understanding Long Videos via LLM-Powered Entity Relation Graphs

arXiv: https://arxiv.org/abs/2501.15953 · HTML: https://mlsanity.com/paper/2501.15953/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2025.

## Built on by (most cited first, 11 of 11)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [ViMax: Agentic Video Generation](https://mlsanity.com/paper/2606.07649/) (2026)
- [SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding](https://mlsanity.com/paper/2506.07600/) (2025)
- [MASR: Self-Reflective Reasoning through Multimodal Hierarchical Attention Focusing for Agent-based Video Understanding](https://mlsanity.com/paper/2504.17213/) (2025)
- [Dynamic Hub-and-Spoke Memory for Streaming Video Understanding](https://mlsanity.com/paper/2608.30294/) (2026)
- [EgoGraph: Temporal Knowledge Graph for Egocentric Video Understanding](https://mlsanity.com/paper/2602.23709/) (2026)
- [MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representation](https://mlsanity.com/paper/2602.07082/) (2026)
- [MINERVA-Cultural: A Benchmark for Cultural and Multilingual Long Video Reasoning](https://mlsanity.com/paper/2601.10649/) (2026)
- [Confidence-Aware Tool Orchestration for Robust Video Understanding](https://mlsanity.com/paper/2606.26904/) (2026)
- [Coarse Indexing, Fine Evidence: Decoupling Temporal Granularity in Long-Video RAG](https://mlsanity.com/paper/2608.23011/) (2026)
- [CAM: Question Answering on Entity-Centric Videos with Continuous Extraction and Adaptive Querying](https://mlsanity.com/paper/2609.06504/) (2026)

## Abstract (arXiv)

The analysis of extended video content poses unique challenges in artificial intelligence, particularly when dealing with the complexity of tracking and understanding visual elements across time. Current methodologies that process video frames sequentially struggle to maintain coherent tracking of objects, especially when these objects temporarily vanish and later reappear in the footage. A critical limitation of these approaches is their inability to effectively identify crucial moments in the video, largely due to their limited grasp of temporal relationships. To overcome these obstacles, we present GraphVideoAgent, a cutting-edge system that leverages the power of graph-based object tracking in conjunction with large language model capabilities. At its core, our framework employs a dynamic graph structure that maps and monitors the evolving relationships between visual entities throughout the video sequence. This innovative approach enables more nuanced understanding of how objects interact and transform over time, facilitating improved frame selection through comprehensive contextual awareness. Our approach demonstrates remarkable effectiveness when tested against industry benchmarks. In evaluations on the EgoSchema dataset, GraphVideoAgent achieved a 2.2 improvement over existing methods while requiring analysis of only 8.2 frames on average. Similarly, testing on the NExT-QA benchmark yielded a 2.0 performance increase with an average frame requirement of 8.1. These results underscore the efficiency of our graph-guided methodology in enhancing both accuracy and computational performance in long-form video understanding tasks.