mlsanityData to 14 Oct 2026

EgoBlind: Towards Egocentric Visual Assistance for the Blind

arXiv 2503.08221 ↗First posted 11 Mar 2025cs.CV cs.AI cs.MMcited by 21

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

We present EgoBlind, the first egocentric VideoQA dataset collected from blind individuals to evaluate the assistive capabilities of contemporary multimodal large language models (MLLMs). EgoBlind comprises 1,392 first-person videos from the daily lives of blind and visually impaired individuals. It also features 5,311 questions directly posed or verified by the blind to reflect their in-situation needs for visual assistance. Each question has an average of 3 manually annotated reference answers to reduce subjectiveness. Using EgoBlind, we comprehensively evaluate 16 advanced MLLMs and find that all models struggle. The best performers achieve an accuracy near 60%, which is far behind human performance of 87.4%. To guide future advancements, we identify and summarize major limitations of existing MLLMs in egocentric visual assistance for the blind and explore heuristic solutions for improvement. With these efforts, we hope that EgoBlind will serve as a foundation for developing effective AI assistants to enhance the independence of the blind and visually impaired. Data and code are available at https://github.com/doc-doc/EgoBlind.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 48 of its 61 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

48 of 61 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024VidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AI15 Oct
  2. 2024MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QA9 Oct
  3. 2023EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models27 Nov

Other references · 45

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  9. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  10. 2015VQA: Visual Question Answeringcited by 2,506
  11. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  12. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
33 more
  1. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  2. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  3. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  4. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  5. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  6. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  7. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  8. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  9. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  10. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  11. 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
  12. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  13. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 546
  14. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 542
  15. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
  16. 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
  17. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  18. 2022Egocentric Video-Language Pretrainingcited by 232
  19. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 217
  20. 2022Learning Video Representations from Large Language Modelscited by 205
  21. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 166
  22. 2023Can I Trust Your Answer? Visually Grounded Video Question Answeringcited by 135
  23. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
  24. 2022EgoTaskQA: Understanding Human Tasks in Egocentric Videoscited by 115
  25. 2025EgoLife: Towards Egocentric Life Assistantcited by 91
  26. 2021Deconfounded Video Moment Retrieval with Causal Interventioncited by 84
  27. 2023An Outlook into the Future of Egocentric Visioncited by 84
  28. 2023Grounded Question-Answering in Long Egocentric Videoscited by 64
  29. 2025EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answeringcited by 44
  30. 2022AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistantcited by 32
  31. 2024VideoQA in the Era of LLMs: An Empirical Studycited by 29
  32. 2024Sycophancy in Vision-Language Models: A Systematic Analysis and an Inference-Time Mitigation Frameworkcited by 17
  33. 2024Video Question Answering for People with Visual Impairments Using an Egocentric 360-Degree Cameracited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 12.

21 papers.

Sort:

By area:

  1. 2025A Survey of Multimodal Hallucination Evaluation and Detectioncited by 27
  2. 2025EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmarkcited by 19
  3. 2025Visual Intention Grounding for Egocentric Assistantscited by 11
  4. 2026EgoSound: Benchmarking Sound Understanding in Egocentric Videoscited by 11
  5. 2026PhoStream: Benchmarking Real-World Streaming for Omnimodal Assistants in Mobile Scenarioscited by 10
  6. 2025Benchmarking Egocentric Multimodal Goal Inference for Assistive Wearable Agentscited by 8
  7. 2026Ego-Grounding for Personalized Question-Answering in Egocentric Videoscited by 7
  8. 2025EgoExo-Con: Exploring View-Invariant Video Temporal Understandingcited by 5
  9. 2025Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challengescited by 5
  10. 2025EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoningcited by 4
  11. 2025SkillSight: Efficient First-Person Skill Assessment with Gazecited by 4
  12. 2026Don't Pause! Every prediction matters in a streaming videocited by 4
9 more
  1. 2026Harnessing Streaming Video in the Wildcited by 4
  2. 2026EgoPro-Bench: Benchmarking Personalized Proactive Interaction in Egocentric Video Streamscited by 3
  3. 2026Say It My Way: Exploring Control in Conversational Visual Question Answering with Blind Userscited by 2
  4. 2026Audio-Visual Intelligence in Large Foundation Modelscited by 2
  5. 2026Do You See What I Am Pointing At? Gesture-Based Egocentric Video Question Answeringcited by 1
  6. 2025Hierarchical Instance Tracking to Balance Privacy Preservation with Accessible Informationcited by 0
  7. 2026X-Stream: Exploring MLLMs as Multiplexers for Multi-Stream Understandingcited by 0
  8. 2026VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistancecited by 0
  9. 2026ManGo: Manga Active Narrative Grounding Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Vision-Language Models for Egocentric Video: From Hand-Object Interaction to Embodied AI
  2. 2025VideoA11y: Method and Dataset for Accessible Video Description
  3. 2025EGOILLUSION: Benchmarking Hallucinations in Egocentric Video Understanding
  4. 2026EgoAVU: Egocentric Audio-Visual Understanding
  5. 2025Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Users
  6. 2024AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding
  7. 2025In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Prompting
  8. 2026How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision People