# "I Can See Forever!": Evaluating Real-time VideoLLMs for Assisting Individuals with Visual Impairments

arXiv: https://arxiv.org/abs/2505.04488 · HTML: https://mlsanity.com/paper/2505.04488/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 7 May 2025.

## Builds on (most cited first, 22 of 22)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://mlsanity.com/paper/2303.04671/) (2023)
- [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023)
- [Jailbreak Attacks and Defenses Against Large Language Models: A Survey](https://mlsanity.com/paper/2407.04295/) (2024)
- [VideoLLM-online: Online Video Large Language Model for Streaming Video](https://mlsanity.com/paper/2406.11816/) (2024)
- [Video Understanding with Large Language Models: A Survey](https://mlsanity.com/paper/2312.17432/) (2023)
- [VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction](https://mlsanity.com/paper/2501.01957/) (2025)
- [LongVLM: Efficient Long Video Understanding via Large Language Models](https://mlsanity.com/paper/2404.03384/) (2024)
- [LITA: Language Instructed Temporal-Localization Assistant](https://mlsanity.com/paper/2403.19046/) (2024)
- [Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles](https://mlsanity.com/paper/2310.08034/) (2023)
- [Instruction Backdoor Attacks Against Customized LLMs](https://mlsanity.com/paper/2402.09179/) (2024)
- [PEFTGuard: Detecting Backdoor Attacks Against Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2411.17453/) (2024)
- [Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People](https://mlsanity.com/paper/2403.15604/) (2024)
- [VIALM: A Survey and Benchmark of Visually Impaired Assistance with Large Models](https://mlsanity.com/paper/2402.01735/) (2024)
- [FC-Attack: Jailbreaking Multimodal Large Language Models via Auto-Generated Flowcharts](https://mlsanity.com/paper/2502.21059/) (2025)
- [Beyond Visual Perception: Insights from Smartphone Interaction of Visually Impaired Users with Large Multimodal Models](https://mlsanity.com/paper/2502.16098/) (2025)
- [Quantized Delta Weight Is Safety Keeper](https://mlsanity.com/paper/2411.19530/) (2024)
- [MISAR: A Multimodal Instructional System with Augmented Reality](https://mlsanity.com/paper/2310.11699/) (2023)
- [WanderGuide: Indoor Map-less Robotic Guide for Exploration by Blind People](https://mlsanity.com/paper/2502.08906/) (2025)

## Built on by (most cited first, 3 of 3)

- [Work Zones challenge VLM Trajectory Planning: Toward Mitigation and Robust Autonomous Driving](https://mlsanity.com/paper/2510.02803/) (2025)
- [Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectification](https://mlsanity.com/paper/2603.24058/) (2026)
- [Do Input-Level Defenses Transfer to Observation-Level Attacks on VideoLLMs?](https://mlsanity.com/paper/2609.08331/) (2026)

## Abstract (arXiv)

The visually impaired population faces significant challenges in daily activities. While prior works employ vision language models for assistance, most focus on static content and cannot address real-time perception needs in complex environments. Recent VideoLLMs enable real-time vision and speech interaction, offering promising potential for assistive tasks. In this work, we conduct the first study evaluating their effectiveness in supporting daily life for visually impaired individuals. We first conducted a user survey with visually impaired participants to design the benchmark VisAssistDaily for daily life evaluation. Using VisAssistDaily, we evaluate popular VideoLLMs and find GPT-4o achieves the highest task success rate. We further conduct a user study to reveal concerns about hazard perception. To address this, we propose SafeVid, an environment-awareness dataset, and fine-tune VITA-1.5, improving risk recognition accuracy from 25.00% to 76.00%.We hope this work provides valuable insights and inspiration for future research in this field.