mlsanityData to 14 Oct 2026

EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answering

arXiv 2502.07411 ↗First posted 11 Feb 2025cs.CV cs.MMcited by 44

Not on the map. Why?

Abstract (arXiv)

We introduce EgoTextVQA, a novel and rigorously constructed benchmark for egocentric QA assistance involving scene text. EgoTextVQA contains 1.5K ego-view videos and 7K scene-text aware questions that reflect real user needs in outdoor driving and indoor house-keeping activities. The questions are designed to elicit identification and reasoning on scene text in an egocentric and dynamic environment. With EgoTextVQA, we comprehensively evaluate 10 prominent multimodal large language models. Currently, all models struggle, and the best results (Gemini 1.5 Pro) are around 33% accuracy, highlighting the severe deficiency of these techniques in egocentric QA assistance. Our further investigations suggest that precise temporal grounding and multi-frame reasoning, along with high resolution and auxiliary scene-text inputs, are key for better performance. With thorough analyses and heuristic suggestions, we hope EgoTextVQA can serve as a solid testbed for research in egocentric scene-text QA assistance. Our dataset is released at: https://github.com/zhousheng97/EgoTextVQA.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Builds on 53 of its 71 references that are in the map. First posted on arXiv 11 Feb 2025.

Builds on

53 of 71 references.

References · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  8. 2020GLU Variants Improve Transformercited by 1,804
  9. 2019Towards VQA Models That Can Readcited by 1,718
  10. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  11. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  12. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
38 more
  1. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  2. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  3. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  4. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  5. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  6. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  7. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  8. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  9. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  10. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  11. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  12. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 476
  13. 2021InfographicVQAcited by 437
  14. 2024InternLM2 Technical Reportcited by 397
  15. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  16. 2019Scene Text Visual Question Answeringcited by 331
  17. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  18. 2022Egocentric Video-Language Pretrainingcited by 232
  19. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 217
  20. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
  21. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 115
  22. 2024Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Modelscited by 113
  23. 2024HourVideo: 1-Hour Video-Language Understandingcited by 113
  24. 2022PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR Systemcited by 106
  25. 2021Deconfounded Video Moment Retrieval with Causal Interventioncited by 84
  26. 2020On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answeringcited by 78
  27. 2024Beyond LLaVA-HD: Diving into High-Resolution Large Multimodal Modelscited by 76
  28. 2022Invariant Grounding for Video Question Answeringcited by 68
  29. 2023Grounded Question-Answering in Long Egocentric Videoscited by 64
  30. 2023EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Modelscited by 60
  31. 2024VidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AIcited by 34
  32. 2022AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistantcited by 32
  33. 2024VideoQA in the Era of LLMs: An Empirical Studycited by 29
  34. 2023Diffusion-based Blind Text Image Super-Resolutioncited by 25
  35. 2024MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QAcited by 24
  36. 2023Reading Between the Lanes: Text VideoQA on the Roadcited by 23
  37. 2024GoMatching: A Simple Baseline for Video Text Spotting via Long and Short Term Matchingcited by 12
  38. 2024LLaVA-Read: Enhancing Reading Ability of Multimodal Language Modelscited by 12

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 23; 2026: 21.

44 papers.

Sort:

By area:

  1. 2025EgoLife: Towards Egocentric Life Assistantcited by 91
  2. 2025In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Promptingcited by 22
  3. 2025EgoBlind: Towards Egocentric Visual Assistance for the Blindcited by 21
  4. 2025EgoCross: Benchmarking Multimodal Large Language Models for Cross-Domain Egocentric Video Question Answeringcited by 21
  5. 2025MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarioscited by 18
  6. 2025EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoTcited by 16
  7. 2025StreamEQA: Towards Streaming Video Understanding for Embodied Scenarioscited by 12
  8. 2025VidText: Towards Comprehensive Evaluation for Video Text Understandingcited by 11
  9. 2025TeleEgo: Benchmarking Egocentric AI Assistants in the Wildcited by 11
  10. 2026EgoSound: Benchmarking Sound Understanding in Egocentric Videoscited by 11
  11. 2025QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generationcited by 9
  12. 2025MICA: Multi-Agent Industrial Coordination Assistantcited by 8
32 more
  1. 2026Ego-Grounding for Personalized Question-Answering in Egocentric Videoscited by 7
  2. 2025Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Ruminationcited by 6
  3. 2026EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understandingcited by 6
  4. 2025Gather and Trace: Rethinking Video TextVQA from an Instance-oriented Perspectivecited by 5
  5. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
  6. 2026LifeEval: A Multimodal Benchmark for Assistive AI in Egocentric Daily Life Taskscited by 5
  7. 2026EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Visioncited by 5
  8. 2025Reasoning under Vision: Understanding Visual-Spatial Cognition in Vision-Language Models for CAPTCHAcited by 4
  9. 2025EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoningcited by 4
  10. 2025SkillSight: Efficient First-Person Skill Assessment with Gazecited by 4
  11. 2025Ego-EXTRA: video-language Egocentric Dataset for EXpert-TRAinee assistancecited by 4
  12. 2026Interaction-aware Representation Modeling with Co-occurrence Consistency for Egocentric Hand-Object Parsingcited by 4
  13. 2025mmWalk: Towards Multi-modal Multi-view Walking Assistancecited by 3
  14. 2026EgoAVU: Egocentric Audio-Visual Understandingcited by 3
  15. 2026Pro2Assist: Continuous Step-aware Proactive Assistance with Multi-modal Egocentric Perception for Long-horizon Procedural Taskscited by 3
  16. 2025Audio Does Matter: Importance-Aware Multi-Granularity Fusion for Video Moment Retrievalcited by 2
  17. 2026Where Vision Becomes Text: Locating the OCR Routing Bottleneck in Vision-Language Modelscited by 2
  18. 2026SafeGuard: A Multi-Agent Perception-Reasoning Framework for Social-Risk AI-Generated Video Detectioncited by 2
  19. 2026T2SGrid: Temporal-to-Spatial Gridification for Video Temporal Groundingcited by 1
  20. 2026EXPLORE-Bench: Egocentric Scene Prediction with Long-Horizon Reasoningcited by 1
  21. 2026Do You See What I Am Pointing At? Gesture-Based Egocentric Video Question Answeringcited by 1
  22. 2026HLL: Can Agents Cross Humanity's Last Line of Verification?cited by 1
  23. 2026ViTexQA: A Multi-Frame Temporal Perception Dataset for Video Text Question Answeringcited by 1
  24. 2025DMC3: Dual-Modal Counterfactual Contrastive Construction for Egocentric Video Question Answeringcited by 0
  25. 2025The N-Body Problem: Parallel Execution from Single-Person Egocentric Videocited by 0
  26. 2025DarkQA: Benchmarking Vision-Language Models on Visual-Primitive Question Answering in Low-Light Indoor Scenescited by 0
  27. 2026CASHEW: Stabilizing Multimodal Reasoning via Iterative Trajectory Aggregationcited by 0
  28. 2026Reinforcing Egocentric Spatial Perception in Multimodal Large Language Models via Ego Scene Augmentationcited by 0
  29. 2026Understanding From Human Perspective: A Multi-agent System for Interactive Egocentric Medical Image Segmentationcited by 0
  30. 2026EgoErrorVQA: Assess Egocentric Comprehension Capabilities through Procedural Errors for Ego-Agentic AIcited by 0
  31. 2026ClearText-Video: A Large-Scale Text-Centric Video Dataset Bridging Video Restoration and Scene-Text Enhancementcited by 0
  32. 2026EgoOmniWorld: Unifying Egocentric Understanding via Ego-Action-World Interactionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Advancing Egocentric Video Question Answering with Multimodal Large Language Models
  2. 2026Graph it first! Enabling Reasoning on Long-form Egocentric Videos through Scene Graphs
  3. 2025Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos
  4. 2026Companion-style QA Assistance in Ego-Vision
  5. 2024HCQA @ Ego4D EgoSchema Challenge 2024
  6. 2026The Right Inference Strategy Is All You Need: Nearly Training-Free Domain-Wise Inference for EgoCross Challenge
  7. 2025EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmark
  8. 2026MA-EgoQA: Question Answering over Egocentric Videos from Multiple Embodied Agents