mlsanityData to 7 Oct 2026

Advancing Egocentric Video Question Answering with Multimodal Large Language Models

arXiv 2504.04550 ↗First posted 6 Apr 2025cs.CV cs.AI cs.LGcited by 8

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Egocentric Video Question Answering (QA) requires models to handle long-horizon temporal reasoning, first-person perspectives, and specialized challenges like frequent camera movement. This paper systematically evaluates both proprietary and open-source Multimodal Large Language Models (MLLMs) on QaEgo4Dv2 - a refined dataset of egocentric videos derived from QaEgo4D. Four popular MLLMs (GPT-4o, Gemini-1.5-Pro, Video-LLaVa-7B and Qwen2-VL-7B-Instruct) are assessed using zero-shot and fine-tuned approaches for both OpenQA and CloseQA settings. We introduce QaEgo4Dv2 to mitigate annotation noise in QaEgo4D, enabling more reliable comparison. Our results show that fine-tuned Video-LLaVa-7B and Qwen2-VL-7B-Instruct achieve new state-of-the-art performance, surpassing previous benchmarks by up to +2.6% ROUGE/METEOR (for OpenQA) and +13% accuracy (for CloseQA). We also present a thorough error analysis, indicating the model's difficulty in spatial reasoning and fine-grained object recognition - key areas for future improvement.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 31 references that are in the map. First posted on arXiv 6 Apr 2025.

Builds on

21 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models27 Nov
  2. 2022From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answering30 May

Other references · 19

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  6. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  7. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  8. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
  9. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  10. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  11. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 559
  12. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
7 more
  1. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  2. 2023LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignmentcited by 366
  3. 2020HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-trainingcited by 362
  4. 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
  5. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 112
  6. 2023Grounded Question-Answering in Long Egocentric Videoscited by 62
  7. 2024MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QAcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2025Object Detection with Multimodal Large Vision-Language Models: An In-depth Reviewcited by 14
  2. 2025PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligencecited by 10
  3. 2025EASG-Bench: Video Q&A Benchmark with Egocentric Action Scene Graphscited by 5
  4. 2025How Far Can Off-the-Shelf Multimodal Large Language Models Go in Online Episodic Memory Question Answering?cited by 1
  5. 2026Do You See What I Am Pointing At? Gesture-Based Egocentric Video Question Answeringcited by 1
  6. 2026Rethinking RAG in Long Videos: What to Retrieve and How to Use It?cited by 1
  7. 2026Graph it first! Enabling Reasoning on Long-form Egocentric Videos through Scene Graphscited by 1
  8. 2026Decoding Pedestrian Crossing Intention from Egocentric Vision via Vision Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025EgoCross: Benchmarking Multimodal Large Language Models for Cross-Domain Egocentric Video Question Answering
  2. 2026The Right Inference Strategy Is All You Need: Nearly Training-Free Domain-Wise Inference for EgoCross Challenge
  3. 2025Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos
  4. 2025EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answering
  5. 2023LifelongMemory: Leveraging LLMs for Answering Queries in Long-form Egocentric Videos
  6. 2026Optimizing Multimodal LLMs for Egocentric Video Understanding: A Solution for the HD-EPIC VQA Challenge
  7. 2025Team of One: Cracking Complex Video QA with Model Synergy
  8. 2024AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding