mlsanityData to 7 Oct 2026

Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos

arXiv 2503.13646 ↗First posted 17 Mar 2025cs.CVcited by 41

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Understanding fine-grained temporal dynamics is crucial in egocentric videos, where continuous streams capture frequent, close-up interactions with objects. In this work, we bring to light that current egocentric video question-answering datasets often include questions that can be answered using only few frames or commonsense reasoning, without being necessarily grounded in the actual video. Our analysis shows that state-of-the-art Multi-Modal Large Language Models (MLLMs) on these benchmarks achieve remarkably high performance using just text or a single frame as input. To address these limitations, we introduce EgoTempo, a dataset specifically designed to evaluate temporal understanding in the egocentric domain. EgoTempo emphasizes tasks that require integrating information across the entire video, ensuring that models would need to rely on temporal patterns rather than static cues or pre-existing knowledge. Extensive experiments on EgoTempo show that current MLLMs still fall short in temporal reasoning on egocentric videos, and thus we hope EgoTempo will catalyze new research in the field and inspire models that better capture the complexity of temporal dynamics. Dataset and code are available at https://github.com/google-research-datasets/egotempo.git.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 32 of its 43 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

32 of 43 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models27 Nov
  2. 2023PaLM-E: An Embodied Multimodal Language Model6 Mar

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  8. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  9. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  10. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  11. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  12. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
18 more
  1. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  2. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  3. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  4. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  5. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  6. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  7. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 352
  8. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  9. 2022Egocentric Video-Language Pretrainingcited by 231
  10. 2019CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoningcited by 137
  11. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 112
  12. 2024Matryoshka Multimodal Modelscited by 82
  13. 2023Grounded Question-Answering in Long Egocentric Videoscited by 62
  14. 2024Koala: Key frame-conditioned long video-LLMcited by 57
  15. 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
  16. 2024How Good is my Video LMM? Complex Video Reasoning and Robustness Evaluation Suite for Video-LMMscited by 30
  17. 2024EAGLE: Egocentric AGgregated Language-video Enginecited by 16
  18. 2024Temporally Grounding Instructional Diagrams in Unconstrained Videoscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 13; 2026: 28.

41 papers.

Sort:

By area:

  1. 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 47
  2. 2025Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMscited by 20
  3. 2025EgoCross: Benchmarking Multimodal Large Language Models for Cross-Domain Egocentric Video Question Answeringcited by 19
  4. 2025EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmarkcited by 16
  5. 2025SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearingcited by 14
  6. 2025CLiViS: Unleashing Cognitive Map through Linguistic-Visual Synergy for Embodied Visual Reasoningcited by 13
  7. 2025Causality Matters: How Temporal Information Emerges in Video Language Modelscited by 9
  8. 2026EgoSound: Benchmarking Sound Understanding in Egocentric Videoscited by 8
  9. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 8
  10. 2025EgoTrigger: Toward Audio-Driven Image Capture for Human Memory Enhancement in All-Day Energy-Efficient Smart Glassescited by 6
  11. 2025EASG-Bench: Video Q&A Benchmark with Egocentric Action Scene Graphscited by 5
  12. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
29 more
  1. 2025Reasoning under Vision: Understanding Visual-Spatial Cognition in Vision-Language Models for CAPTCHAcited by 4
  2. 2025EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoningcited by 4
  3. 2026Interaction-aware Representation Modeling with Co-occurrence Consistency for Egocentric Hand-Object Parsingcited by 4
  4. 2026EgoGraph: Temporal Knowledge Graph for Egocentric Video Understandingcited by 3
  5. 2026Video-Oasis: Rethinking Evaluation of Video Understandingcited by 3
  6. 2026EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Visioncited by 3
  7. 2025HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenescited by 2
  8. 2026EgoAVU: Egocentric Audio-Visual Understandingcited by 2
  9. 2026Can Vision-Language Models Solve the Shell Game?cited by 2
  10. 2026ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Modelscited by 2
  11. 2026Leveraging Gaze and Set-of-Mark in VLLMs for Human-Object Interaction Anticipation from Egocentric Videoscited by 2
  12. 2026Exploring Audio Hallucination in Egocentric Video Understandingcited by 1
  13. 2026Task-Focused Memorization for Multimodal Agentscited by 1
  14. 2026VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understandingcited by 1
  15. 2026Graph it first! Enabling Reasoning on Long-form Egocentric Videos through Scene Graphscited by 1
  16. 2026S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrievalcited by 1
  17. 2025Training-free Online Video Step Groundingcited by 0
  18. 2026Incentivizing Temporal-Awareness in Egocentric Video Understanding Modelscited by 0
  19. 2026SiMing-Bench: Evaluating Procedural Correctness from Continuous Interactions in Clinical Skill Videoscited by 0
  20. 2026Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understandingcited by 0
  21. 2026From Frames to Temporal Graphs: In-Context Egocentric Action Recognition with Vision-Language Modelscited by 0
  22. 2026P-JEPA: Procedural Video Representation Learning via Joint Embedding Predictive Architecturecited by 0
  23. 2026Understanding From Human Perspective: A Multi-agent System for Interactive Egocentric Medical Image Segmentationcited by 0
  24. 2026The First EgoCross Challenge at EgoVis 2026: Cross-Domain Egocentric Video Question Answeringcited by 0
  25. 2026Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligencecited by 0
  26. 2026ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Editscited by 0
  27. 2026DementiaCare-Bench: A Modality-Validated Video Benchmarkcited by 0
  28. 2026Does Local Video Understanding Transfer Across Encounters? The EgoGears Benchmarkcited by 0
  29. 2026EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videoscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understanding
  2. 2025Advancing Egocentric Video Question Answering with Multimodal Large Language Models
  3. 2025EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoT
  4. 2026EgoCoT-Bench: Benchmarking Grounded and Verifiable Operation-Centric Chain of Thought Reasoning for MLLMs
  5. 2026EgoMonth: A Month-Level Egocentric Video Benchmark for Long-Term Spatiotemporal Memory
  6. 2026MA-EgoQA: Question Answering over Egocentric Videos from Multiple Embodied Agents
  7. 2026The Right Inference Strategy Is All You Need: Nearly Training-Free Domain-Wise Inference for EgoCross Challenge
  8. 2026EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding