# VisionAssist: An Open-Source Smartphone Assistant for AI-Based Visual Accessibility

arXiv: https://arxiv.org/abs/2607.16750 · HTML: https://mlsanity.com/paper/2607.16750/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 9 references that are in the map.
- First posted on arXiv 18 Jul 2026.

## Builds on (most cited first, 5 of 5)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [Enhancing the Travel Experience for People with Visual Impairments through Multimodal Interaction: NaviGPT, A Real-Time AI-Driven Mobile Navigation System](https://mlsanity.com/paper/2410.04005/) (2024)
- [NaviSense: A Multimodal Assistive Mobile application for Object Retrieval by Persons with Visual Impairment](https://mlsanity.com/paper/2509.18672/) (2025)

## Abstract (arXiv)

People with low vision often face challenges in performing everyday tasks that require interpreting visual information. We present VisionAssist, an open-source mobile application designed to improve independence by providing AI-powered visual assistance through a smartphone. The application integrates three complementary functionalities within a single interface. First, it enables users to locate specific objects by analyzing the live camera feed. Second, it generates spoken descriptions of captured images, allowing users to identify visual content such as food labels, documents, and everyday objects. Third, it integrates with the smartphone's contacts and calendar to facilitate emergency calls and provide voice-based reminders. The application supports hands-free interaction through voice commands and delivers all feedback using text-to-speech synthesis, making it fully accessible to users with visual impairments. By combining multiple assistive services into a unified platform and releasing the project as open-source software, the proposed solution aims to encourage community contributions and accelerate the development of accessible technologies. The source code is publicly available at: https://github.com/AOzlemC/LowVisionProject.git