# Beyond Descriptions: A Generative Scene2Audio Framework for Blind and Low-Vision Users to Experience Vista Landscapes

arXiv: https://arxiv.org/abs/2603.27295 · HTML: https://mlsanity.com/paper/2603.27295/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 81 references that are in the map.
- First posted on arXiv 28 Mar 2026.

## Builds on (most cited first, 9 of 9)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [I Hear Your True Colors: Image Guided Audio Generation](https://mlsanity.com/paper/2211.03089/) (2022)
- [Design Principles for Generative AI Applications](https://mlsanity.com/paper/2401.14484/) (2024)
- [Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People](https://mlsanity.com/paper/2403.15604/) (2024)
- [PicoAudio: Enabling Precise Timestamp and Frequency Controllability of Audio Events in Text-to-audio Generation](https://mlsanity.com/paper/2407.02869/) (2024)
- [Understanding Visual Arts Experiences of Blind People](https://mlsanity.com/paper/2301.12687/) (2023)
- [MorphFader: Enabling Fine-grained Controllable Morphing with Text-to-Audio Models](https://mlsanity.com/paper/2408.07260/) (2024)
- [Example-Based Framework for Perceptually Guided Audio Texture Generation](https://mlsanity.com/paper/2308.11859/) (2023)

## Abstract (arXiv)

Current scene perception tools for Blind and Low Vision (BLV) individuals rely on spoken descriptions but lack engaging representations of visually pleasing distant environmental landscapes (Vista spaces). Our proposed Scene2Audio framework generates comprehensible and enjoyable nonverbal audio using generative models informed by psychoacoustics, and principles of scene audio composition. Through a user study with 11 BLV participants, we found that combining the Scene2Audio sounds with speech creates a better experience than speech alone, as the sound effects complement the speech making the scene easier to imagine. A mobile app "in-the-wild" study with 7 BLV users for more than a week further showed the potential of Scene2Audio in enhancing outdoor scene experiences. Our work bridges the gap between visual and auditory scene perception by moving beyond purely descriptive aids, addressing the aesthetic needs of BLV users.