mlsanityData to 14 Oct 2026

Grounded Question-Answering in Long Egocentric Videos

arXiv 2312.06505 ↗First posted 11 Dec 2023cs.CVcited by 64

Not on the map. Why?

Abstract (arXiv)

Existing approaches to video understanding, mainly designed for short videos from a third-person perspective, are limited in their applicability in certain fields, such as robotics. In this paper, we delve into open-ended question-answering (QA) in long, egocentric videos, which allows individuals or robots to inquire about their own past visual experiences. This task presents unique challenges, including the complexity of temporally grounding queries within extensive video content, the high resource demands for precise data annotation, and the inherent difficulty of evaluating open-ended answers due to their ambiguous nature. Our proposed approach tackles these challenges by (i) integrating query grounding and answering within a unified model to reduce error propagation; (ii) employing large language models for efficient and scalable data synthesis; and (iii) introducing a close-ended QA task for evaluation, to manage answer ambiguity. Extensive experiments demonstrate the effectiveness of our method, which also achieves state-of-the-art performance on the QaEgo4D and Ego4D-NLQ benchmarks. Code, data, and models are available at https://github.com/Becomebright/GroundVQA.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). Builds on 40 of its 57 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

40 of 57 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  7. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  8. 2018SlowFast Networks for Video Recognitioncited by 1,867
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  10. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  11. 2020Object-Centric Learning with Slot Attentioncited by 876
  12. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 803
28 more
  1. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  2. 2017Dense-Captioning Events in Videoscited by 793
  3. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  4. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  5. 2017Localizing Moments in Video with Natural Languagecited by 663
  6. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  7. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  8. 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 459
  9. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  10. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 311
  11. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275
  12. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  13. 2020Span-based Localizing Network for Natural Language Video Localizationcited by 235
  14. 2022Egocentric Video-Language Pretrainingcited by 232
  15. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  16. 2021Anticipative Video Transformercited by 200
  17. 2023UniVTG: Towards Unified Video-Language Temporal Groundingcited by 190
  18. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 166
  19. 2018Grounded Human-Object Interaction Hotspots from Videocited by 129
  20. 2021Learning to Track with Object Permanencecited by 119
  21. 2018Actor and Observer: Joint Modeling of First and Third-Person Videoscited by 103
  22. 2023UnLoc: A Unified Framework for Video Localization Taskscited by 66
  23. 2022InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challengescited by 48
  24. 2023NaQ: Leveraging Narrations as Queries to Supervise Episodic Memorycited by 46
  25. 2023Helping Hands: An Object-Aware Ego-Centric Video Recognition Modelcited by 31
  26. 2023GroundNLQ @ Ego4D Natural Language Queries Challenge 2023cited by 20
  27. 2022ReLER@ZJU-Alibaba Submission to the Ego4D Natural Language Queries Challenge 2022cited by 19
  28. 2021Learning to Rehearse in Long Sequence Memorizationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 36; 2026: 21.

64 papers.

Sort:

By area:

  1. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 127
  2. 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 117
  3. 2024E.T. Bench: Towards Open-Ended Event-Level Video-Language Understandingcited by 70
  4. 2025Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoningcited by 68
  5. 2025StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistantcited by 63
  6. 2025SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understandingcited by 57
  7. 2025StreamMind: Unlocking Full Frame Rate Streaming Video Dialogue through Event-Gated Cognitioncited by 46
  8. 2024EgoVideo: Exploring Egocentric Foundation Model and Downstream Adaptationcited by 44
  9. 2025EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answeringcited by 44
  10. 2025Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videoscited by 44
  11. 2025ViSpeak: Visual Instruction Feedback in Streaming Videoscited by 42
  12. 2025VideoITG: Multimodal Video Understanding with Instructed Temporal Groundingcited by 29
38 more
  1. 2025Streaming Video Instruction Tuningcited by 26
  2. 2025Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understandingcited by 25
  3. 2024MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QAcited by 24
  4. 2025EgoBlind: Towards Egocentric Visual Assistance for the Blindcited by 21
  5. 2025A Survey on Video Temporal Grounding with Multimodal Large Language Modelcited by 20
  6. 2025Building a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMscited by 19
  7. 2025Object-centric Video Question Answering with Visual Grounding and Referringcited by 13
  8. 2025AdsQA: Towards Advertisement Video Understandingcited by 13
  9. 2024Human-inspired Perspectives: A Survey on AI Long-term Memorycited by 12
  10. 2025Object-Shot Enhanced Grounding Network for Egocentric Videocited by 11
  11. 2025Fine-grained Spatiotemporal Grounding on Egocentric Videoscited by 11
  12. 2023Multi-Sentence Grounding for Long-term Instructional Videocited by 10
  13. 2025EGOILLUSION: Benchmarking Hallucinations in Egocentric Video Understandingcited by 10
  14. 2025VideoExpert: Augmented LLM for Temporal-Sensitive Video Understandingcited by 9
  15. 2025AdaRD-key: Adaptive Relevance-Diversity Keyframe Sampling for Long-form Video understandingcited by 9
  16. 2025Advancing Egocentric Video Question Answering with Multimodal Large Language Modelscited by 8
  17. 2026Ego-Grounding for Personalized Question-Answering in Egocentric Videoscited by 7
  18. 2025KFS-Bench: Comprehensive Evaluation of Key Frame Sampling in Long Video Understandingcited by 6
  19. 2025ANNEXE: Unified Analyzing, Answering, and Pixel Grounding for Egocentric Interactioncited by 5
  20. 2025ViQAgent: Zero-Shot Video Question Answering via Agent with Open-Vocabulary Grounding Validationcited by 5
  21. 2025EgoExo-Con: Exploring View-Invariant Video Temporal Understandingcited by 5
  22. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
  23. 2026EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Visioncited by 5
  24. 2026StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMscited by 5
  25. 2024Episodic Memory Verbalization using Hierarchical Representations of Life-Long Robot Experiencecited by 4
  26. 2025Factorized Learning for Temporally Grounded Video-Language Modelscited by 4
  27. 2025Egocentric Human-Object Interaction Detection: A New Benchmark and Methodcited by 3
  28. 2026EgoAVU: Egocentric Audio-Visual Understandingcited by 3
  29. 2026Egocentric Co-Pilot: Web-Native Smart-Glasses Agents for Assistive Egocentric AIcited by 3
  30. 2026Personal Visual Context Learning in Large Multimodal Modelscited by 3
  31. 2025Measure Twice, Cut Once: A Semantic-Oriented Approach to Video Temporal Localization with Video LLMscited by 2
  32. 2026VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understandingcited by 2
  33. 2024Infusing Environmental Captions for Long-Form Video Language Groundingcited by 1
  34. 2025How Far Can Off-the-Shelf Multimodal Large Language Models Go in Online Episodic Memory Question Answering?cited by 1
  35. 2025ProcObject-10K: Benchmarking Object-Centric Procedural Understanding in Instructional Videoscited by 1
  36. 2026Exploring Multimodal LMMs for Online Episodic Memory Question Answering on the Edgecited by 1
  37. 2026Do You See What I Am Pointing At? Gesture-Based Egocentric Video Question Answeringcited by 1
  38. 2026MARS: Technical Report for the CASTLE Challenge at EgoVis 2026cited by 1

50 most cited of 64 All 64 (JSON)