mlsanityData to 7 Oct 2026

Ego-Grounding for Personalized Question-Answering in Egocentric Videos

arXiv 2604.01966 ↗First posted 2 Apr 2026cs.CV cs.AI cs.ROcited by 6

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

We present the first systematic analysis of multimodal large language models (MLLMs) in personalized question-answering requiring ego-grounding - the ability to understand the camera-wearer in egocentric videos. To this end, we introduce MyEgo, the first egocentric VideoQA dataset designed to evaluate MLLMs' ability to understand, remember, and reason about the camera wearer. MyEgo comprises 541 long videos and 5K personalized questions asking about "my things", "my activities", and "my past". Benchmarking reveals that competitive MLLMs across variants, including open-source vs. proprietary, thinking vs. non-thinking, small vs. large scales all struggle on MyEgo. Top closed- and open-source models (e.g., GPT-5 and Qwen3-VL) achieve only~46% and 36% accuracy, trailing human performance by near 40% and 50% respectively. Surprisingly, neither explicit reasoning nor model scaling yield consistent improvements. Models improve when relevant evidence is explicitly provided, but gains drop over time, indicating limitations in tracking and remembering "me" and "my past". These findings collectively highlight the crucial role of ego-grounding and long-range memory in enabling personalized QA in egocentric videos. We hope MyEgo and our analyses catalyze further progress in these areas for egocentric personalized assistance. Data and code are available at https://github.com/Ryougetsu3606/MyEgo

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 56 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

50 of 56 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challenges17 Nov
  2. 2025EgoBlind: Towards Egocentric Visual Assistance for the Blind11 Mar
  3. 2023EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models27 Nov

Other references · 47

  1. 2025Qwen2.5-VL Technical Reportcited by 5,418
  2. 2024GPT-4o System Cardcited by 4,467
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  6. 2025Qwen3-VL Technical Reportcited by 2,534
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  8. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  9. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  10. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  11. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  12. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
35 more
  1. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  2. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  3. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 666
  4. 2024Long Context Transfer from Language to Visioncited by 502
  5. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understandingcited by 341
  6. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
  7. 2022Egocentric Video-Language Pretrainingcited by 231
  8. 2022Learning Video Representations from Large Language Modelscited by 204
  9. 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 174
  10. 2024Streaming Long Video Understanding with Large Language Modelscited by 164
  11. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 159
  12. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
  13. 2025Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reactioncited by 122
  14. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 112
  15. 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 112
  16. 2022"This is my unicorn, Fluffy": Personalizing frozen vision-language representationscited by 95
  17. 2023An Outlook into the Future of Egocentric Visioncited by 84
  18. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
  19. 2023Grounded Question-Answering in Long Egocentric Videoscited by 62
  20. 2024MyVLM: Personalizing VLMs for User-Specific Queriescited by 58
  21. 2022In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimationcited by 51
  22. 2025EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answeringcited by 43
  23. 2024Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objectscited by 35
  24. 2024AMEGO: Active Memory from long EGOcentric videoscited by 35
  25. 2021Self-Regulated Learning for Egocentric Video Activity Anticipationcited by 32
  26. 2024Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mindcited by 28
  27. 2023RefEgo: Referring Expression Comprehension Dataset from First-Person Perception of Ego4Dcited by 21
  28. 2023Meta-Personalizing Vision-Language Models to Find Named Instances in Videocited by 16
  29. 2025The CASTLE 2024 Dataset: Advancing the Art of Multimodal Understandingcited by 15
  30. 2023EgoPCA: A New Framework for Egocentric Hand-Object Interaction Understandingcited by 14
  31. 2025PVChat: Personalized Video Chat with One-Shot Learningcited by 13
  32. 2025Visual Intention Grounding for Egocentric Assistantscited by 11
  33. 2025TeleEgo: Benchmarking Egocentric AI Assistants in the Wildcited by 9
  34. 2025Mitigating Easy Option Bias in Multiple-Choice Question Answeringcited by 4
  35. 2025Causal-Entity Reflected Egocentric Traffic Accident Video Synthesiscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025EgoCross: Benchmarking Multimodal Large Language Models for Cross-Domain Egocentric Video Question Answering
  2. 2025Advancing Egocentric Video Question Answering with Multimodal Large Language Models
  3. 2025Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos
  4. 2025EgoToM: Benchmarking Theory of Mind Reasoning from Egocentric Videos
  5. 2024MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QA
  6. 2026The Right Inference Strategy Is All You Need: Nearly Training-Free Domain-Wise Inference for EgoCross Challenge
  7. 2024Grounded Multi-Hop VideoQA in Long-Form Egocentric Videos
  8. 2025Ego-EXTRA: video-language Egocentric Dataset for EXpert-TRAinee assistance