# Video Question Answering for People with Visual Impairments Using an Egocentric 360-Degree Camera

arXiv: https://arxiv.org/abs/2405.19794 · HTML: https://mlsanity.com/paper/2405.19794/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 30 May 2024.

## Builds on (most cited first, 19 of 19)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [VizWiz Grand Challenge: Answering Visual Questions from Blind People](https://mlsanity.com/paper/1802.08218/) (2018)
- [NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions](https://mlsanity.com/paper/2105.08276/) (2021)
- [ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering](https://mlsanity.com/paper/1906.02467/) (2019)
- [TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering](https://mlsanity.com/paper/1704.04497/) (2017)
- [Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering](https://mlsanity.com/paper/1904.04357/) (2019)
- [EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone](https://mlsanity.com/paper/2307.05463/) (2023)
- [EgoTaskQA: Understanding Human Tasks in Egocentric Videos](https://mlsanity.com/paper/2210.03929/) (2022)
- [A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input](https://mlsanity.com/paper/1410.0210/) (2014)
- [Motion-Appearance Co-Memory Networks for Video Question Answering](https://mlsanity.com/paper/1803.10906/) (2018)
- [Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videos](https://mlsanity.com/paper/2110.05122/) (2021)
- [Why Does a Visual Question Have Different Answers?](https://mlsanity.com/paper/1908.04342/) (2019)
- [Episodic Memory Question Answering](https://mlsanity.com/paper/2205.01652/) (2022)
- [Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy](https://mlsanity.com/paper/2402.07270/) (2024)
- [Watching the News: Towards VideoQA Models that can Read](https://mlsanity.com/paper/2211.05588/) (2022)
- [Vision Skills Needed to Answer Visual Questions](https://mlsanity.com/paper/2010.03160/) (2020)
- [Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts](https://mlsanity.com/paper/2309.15915/) (2023)

## Built on by (most cited first, 9 of 9)

- [Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning?](https://mlsanity.com/paper/2505.11907/) (2025)
- [EgoBlind: Towards Egocentric Visual Assistance for the Blind](https://mlsanity.com/paper/2503.08221/) (2025)
- [PANORAMA: The Rise of Omnidirectional Vision in the Embodied AI Era](https://mlsanity.com/paper/2509.12989/) (2025)
- [Anomaly Detection for People with Visual Impairments Using an Egocentric 360-Degree Camera](https://mlsanity.com/paper/2411.10945/) (2024)
- [Building Egocentric Procedural AI Assistant: Methods, Benchmarks, and Challenges](https://mlsanity.com/paper/2511.13261/) (2025)
- [Bounding-Box Trajectories Matter for Video Anomaly Detection](https://mlsanity.com/paper/2605.21957/) (2026)
- [Rethinking Open-World Video Anomaly Detection: Diagnosing Definition Blindness](https://mlsanity.com/paper/2607.20780/) (2026)
- [Enhancing Screen Time Identification in Children with a Multi-View Vision Language Model and Screen Time Tracker](https://mlsanity.com/paper/2410.01966/) (2024)
- [VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistance](https://mlsanity.com/paper/2607.14660/) (2026)

## Abstract (arXiv)

This paper addresses the daily challenges encountered by visually impaired individuals, such as limited access to information, navigation difficulties, and barriers to social interaction. To alleviate these challenges, we introduce a novel visual question answering dataset. Our dataset offers two significant advancements over previous datasets: Firstly, it features videos captured using a 360-degree egocentric wearable camera, enabling observation of the entire surroundings, departing from the static image-centric nature of prior datasets. Secondly, unlike datasets centered on singular challenges, ours addresses multiple real-life obstacles simultaneously through an innovative visual-question answering framework. We validate our dataset using various state-of-the-art VideoQA methods and diverse metrics. Results indicate that while progress has been made, satisfactory performance levels for AI-powered assistive services remain elusive for visually impaired individuals. Additionally, our evaluation highlights the distinctive features of the proposed dataset, featuring ego-motion in videos captured via 360-degree cameras across varied scenarios.