# Scene-Aware Audio for 360textdegree Videos

arXiv: https://arxiv.org/abs/1805.04792 · HTML: https://mlsanity.com/paper/1805.04792/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2018.

## Built on by (most cited first, 25 of 25)

- [AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesis](https://mlsanity.com/paper/2302.02088/) (2023)
- [Visually Informed Binaural Audio Generation without Binaural Audios](https://mlsanity.com/paper/2104.06162/) (2021)
- [Learning in Audio-visual Context: A Review, Analysis, and New Perspective](https://mlsanity.com/paper/2208.09579/) (2022)
- [Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis](https://mlsanity.com/paper/2103.14201/) (2021)
- [Learning Audio-Visual Dereverberation](https://mlsanity.com/paper/2106.07732/) (2021)
- [Neural Acoustic Context Field: Rendering Realistic Room Impulse Response With Neural Fields](https://mlsanity.com/paper/2309.15977/) (2023)
- [FoleyGAN: Visually Guided Generative Adversarial Network-Based Synchronous Sound Generation in Silent Videos](https://mlsanity.com/paper/2107.09262/) (2021)
- [Exploiting Audio-Visual Consistency with Partial Supervision for Spatial Audio Generation](https://mlsanity.com/paper/2105.00708/) (2021)
- [Modeling and Driving Human Body Soundfields through Acoustic Primitives](https://mlsanity.com/paper/2407.13083/) (2024)
- [Listen2Scene: Interactive material-aware binaural sound propagation for reconstructed 3D scenes](https://mlsanity.com/paper/2302.02809/) (2023)
- [SEE-2-SOUND: Zero-Shot Spatial Environment-to-Spatial Sound](https://mlsanity.com/paper/2406.06612/) (2024)
- [LAVSS: Location-Guided Audio-Visual Spatial Audio Separation](https://mlsanity.com/paper/2310.20446/) (2023)
- [Hearing Anywhere in Any Environment](https://mlsanity.com/paper/2504.10746/) (2025)
- [Sounding Bodies: Modeling 3D Spatial Sound of Humans Using Body Pose and Audio](https://mlsanity.com/paper/2311.06285/) (2023)
- [Differentiable Room Acoustic Rendering with Multi-View Vision Priors](https://mlsanity.com/paper/2504.21847/) (2025)
- [Binaural SoundNet: Predicting Semantics, Depth and Motion with Binaural Sounds](https://mlsanity.com/paper/2109.02763/) (2021)
- [How Would It Sound? Material-Controlled Multimodal Acoustic Profile Generation for Indoor Scenes](https://mlsanity.com/paper/2508.02905/) (2025)
- [SOAF: Scene Occlusion-aware Neural Acoustic Field](https://mlsanity.com/paper/2407.02264/) (2024)
- [SonoWorld: From One Image to a 3D Audio-Visual Scene](https://mlsanity.com/paper/2603.28757/) (2026)
- [Provable Robustness for Streaming Models with a Sliding Window](https://mlsanity.com/paper/2303.16308/) (2023)
- [Cross-modal Generative Model for Visual-Guided Binaural Stereo Generation](https://mlsanity.com/paper/2311.07630/) (2023)
- [In-the-wild Audio Spatialization with Flexible Text-guided Localization](https://mlsanity.com/paper/2506.00927/) (2025)
- [Dynamic Multi-Species Bird Soundscape Generation with Acoustic Patterning and 3D Spatialization](https://mlsanity.com/paper/2511.19275/) (2025)
- [Materialistic RIR: Material Conditioned Realistic RIR Generation](https://mlsanity.com/paper/2604.21119/) (2026)
- [Audible World Models: Spatially Aware Sound Generation for 3D Worlds](https://mlsanity.com/paper/2609.38444/) (2026)

## Abstract (arXiv)

Although 360textdegree cameras ease the capture of panoramic footage, it remains challenging to add realistic 360textdegree audio that blends into the captured scene and is synchronized with the camera motion. We present a method for adding scene-aware spatial audio to 360textdegree videos in typical indoor scenes, using only a conventional mono-channel microphone and a speaker. We observe that the late reverberation of a room's impulse response is usually diffuse spatially and directionally. Exploiting this fact, we propose a method that synthesizes the directional impulse response between any source and listening locations by combining a synthesized early reverberation part and a measured late reverberation tail. The early reverberation is simulated using a geometric acoustic simulation and then enhanced using a frequency modulation method to capture room resonances. The late reverberation is extracted from a recorded impulse response, with a carefully chosen time duration that separates out the late reverberation from the early reverberation. In our validations, we show that our synthesized spatial audio matches closely with recordings using ambisonic microphones. Lastly, we demonstrate the strength of our method in several applications.