# X-LeBench: A Benchmark for Extremely Long Egocentric Video Understanding

arXiv: https://arxiv.org/abs/2501.06835 · HTML: https://mlsanity.com/paper/2501.06835/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 12 Jan 2025.

## Builds on (most cited first, 30 of 34)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions](https://mlsanity.com/paper/2105.08276/) (2021)
- [EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding](https://mlsanity.com/paper/2308.09126/) (2023)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language](https://mlsanity.com/paper/2204.00598/) (2022)
- [Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives](https://mlsanity.com/paper/2311.18259/) (2023)
- [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024)
- [HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training](https://mlsanity.com/paper/2005.00200/) (2020)
- [LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding](https://mlsanity.com/paper/2410.17434/) (2024)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [Perception Test: A Diagnostic Benchmark for Multimodal Video Models](https://mlsanity.com/paper/2305.13786/) (2023)
- [Span-based Localizing Network for Natural Language Video Localization](https://mlsanity.com/paper/2004.13931/) (2020)
- [EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone](https://mlsanity.com/paper/2307.05463/) (2023)
- [EgoTaskQA: Understanding Human Tasks in Egocentric Videos](https://mlsanity.com/paper/2210.03929/) (2022)
- [CinePile: A Long Video Question Answering Dataset and Benchmark](https://mlsanity.com/paper/2405.08813/) (2024)
- [HourVideo: 1-Hour Video-Language Understanding](https://mlsanity.com/paper/2411.04998/) (2024)
- [Video ReCap: Recursive Captioning of Hour-Long Videos](https://mlsanity.com/paper/2402.13250/) (2024)
- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)

## Built on by (most cited first, 15 of 15)

- [ST-Think: How Multimodal Large Language Models Reason About 4D Worlds from Ego-Centric Videos](https://mlsanity.com/paper/2503.12542/) (2025)
- [Towards Online Multi-Modal Social Interaction Understanding](https://mlsanity.com/paper/2503.19851/) (2025)
- [TeleEgo: Benchmarking Egocentric AI Assistants in the Wild](https://mlsanity.com/paper/2510.23981/) (2025)
- [Benchmarking Egocentric Multimodal Goal Inference for Assistive Wearable Agents](https://mlsanity.com/paper/2510.22443/) (2025)
- [M3Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Tasks](https://mlsanity.com/paper/2606.05008/) (2026)
- [S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval](https://mlsanity.com/paper/2607.02689/) (2026)
- [EXPLORE-Bench: Egocentric Scene Prediction with Long-Horizon Reasoning](https://mlsanity.com/paper/2603.09731/) (2026)
- [EgoIntent: A Pre-Outcome Micro-Step Benchmark for Understanding What, Why, and Next](https://mlsanity.com/paper/2603.12147/) (2026)
- [Rethinking RAG in Long Videos: What to Retrieve and How to Use It?](https://mlsanity.com/paper/2606.13141/) (2026)
- [EgoEverything: A Benchmark for Human Behavior Inspired Long Context Egocentric Video Understanding in AR Environment](https://mlsanity.com/paper/2604.08342/) (2026)
- [TeachObs: A Human-Validated Benchmark for Multimodal Teaching Observation and Model Evaluation](https://mlsanity.com/paper/2605.30673/) (2026)
- [LongEgoRefer: A Benchmark for Long-Form Egocentric Video Referring Expression Comprehension](https://mlsanity.com/paper/2607.02096/) (2026)
- [SCOUT: Self-Checking and Recovery-Aware Tool-Thought Agents for Ultra-Long Egocentric Video Reasoning](https://mlsanity.com/paper/2608.07959/) (2026)
- [To Memories and Beyond: From Remembering to Knowing You across Long-Term Multimodal Personal Archives](https://mlsanity.com/paper/2609.19167/) (2026)
- [EMBER-Bench: Benchmarking Cross-Event Causal Memory in Long-Horizon Embodied Tasks](https://mlsanity.com/paper/2610.05013/) (2026)

## Abstract (arXiv)

Long-form egocentric video understanding provides rich contextual information and unique insights into long-term human behaviors, holding significant potential for applications in embodied intelligence, long-term activity analysis, and personalized assistive technologies. However, existing benchmark datasets primarily focus on single, short (eg, minutes to tens of minutes) to moderately long videos, leaving a substantial gap in evaluating extensive, ultra-long egocentric video recordings. To address this, we introduce X-LeBench, a novel benchmark dataset meticulously designed to fill this gap by focusing on tasks requiring a comprehensive understanding of extremely long egocentric video recordings. Our X-LeBench develops a life-logging simulation pipeline that produces realistic, coherent daily plans aligned with real-world video data. This approach enables the flexible integration of synthetic daily plans with real-world footage from Ego4D-a massive-scale egocentric video dataset covers a wide range of daily life scenarios-resulting in 432 simulated video life logs spanning from 23 minutes to 16.4 hours. The evaluations of several baseline systems and multimodal large language models (MLLMs) reveal their poor performance across the board, highlighting the inherent challenges of long-form egocentric video understanding, such as temporal localization and reasoning, context aggregation, and memory retention, and underscoring the need for more advanced models.