# SonifyAR: Context-Aware Sound Generation in Augmented Reality

arXiv: https://arxiv.org/abs/2405.07089 · HTML: https://mlsanity.com/paper/2405.07089/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2024.

## Built on by (most cited first, 5 of 5)

- [StreetReaderAI: Making Street View Accessible Using Context-Aware Multimodal AI](https://mlsanity.com/paper/2508.08524/) (2025)
- [PAVAS: Physics-Aware Video-to-Audio Synthesis](https://mlsanity.com/paper/2512.08282/) (2025)
- [Sonify Anything: Towards Context-Aware Sonic Interactions in AR](https://mlsanity.com/paper/2508.01789/) (2025)
- [Generative Augmented Reality: Paradigms, Technologies, and Future Applications](https://mlsanity.com/paper/2511.16783/) (2025)
- [MoXaRt: Audio-Visual Object-Guided Sound Interaction for XR](https://mlsanity.com/paper/2603.10465/) (2026)

## Abstract (arXiv)

Sound plays a crucial role in enhancing user experience and immersiveness in Augmented Reality (AR). However, current platforms lack support for AR sound authoring due to limited interaction types, challenges in collecting and specifying context information, and difficulty in acquiring matching sound assets. We present SonifyAR, an LLM-based AR sound authoring system that generates context-aware sound effects for AR experiences. SonifyAR expands the current design space of AR sound and implements a Programming by Demonstration (PbD) pipeline to automatically collect contextual information of AR events, including virtual content semantics and real world context. This context information is then processed by a large language model to acquire sound effects with Recommendation, Retrieval, Generation, and Transfer methods. To evaluate the usability and performance of our system, we conducted a user study with eight participants and created five example applications, including an AR-based science experiment, an improving case for AR headset safety, and an assisting example for low vision AR users.