# MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenes

arXiv: https://arxiv.org/abs/2205.09248 · HTML: https://mlsanity.com/paper/2205.09248/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 80 references that are in the map.
- First posted on arXiv 18 May 2022.

## Builds on (most cited first, 30 of 35)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [Graph U-Nets](https://mlsanity.com/paper/1905.05178/) (2019)
- [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020)
- [StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1710.10916/) (2017)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [Understanding Attention and Generalization in Graph Neural Networks](https://mlsanity.com/paper/1905.02850/) (2019)
- [LibriMix: An Open-Source Dataset for Generalizable Speech Separation](https://mlsanity.com/paper/2005.11262/) (2020)
- [Scan2CAD: Learning CAD Model Alignment in RGB-D Scans](https://mlsanity.com/paper/1811.11187/) (2018)
- [Visually Indicated Sounds](https://mlsanity.com/paper/1512.08512/) (2015)
- [Learning Neural Acoustic Fields](https://mlsanity.com/paper/2204.00628/) (2022)
- [A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions](https://mlsanity.com/paper/2011.06801/) (2020)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Asteroid: the PyTorch-based audio source separation toolkit for researchers](https://mlsanity.com/paper/2005.04132/) (2020)
- [Visual Acoustic Matching](https://mlsanity.com/paper/2202.06875/) (2022)
- [Building and Evaluation of a Real Room Impulse Response Dataset](https://mlsanity.com/paper/1811.06795/) (2018)
- [FAST-RIR: Fast neural diffuse room impulse response generator](https://mlsanity.com/paper/2110.04057/) (2021)
- [Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis](https://mlsanity.com/paper/2103.14201/) (2021)
- [Voices Obscured in Complex Environmental Settings (VOICES) corpus](https://mlsanity.com/paper/1804.05053/) (2018)
- [GWA: A Large High-Quality Acoustic Dataset for Audio Processing](https://mlsanity.com/paper/2204.01787/) (2022)
- [The Cone of Silence: Speech Separation by Localization](https://mlsanity.com/paper/2010.06007/) (2020)
- [Deep Impulse Responses: Estimating and Parameterizing Filters with Deep Networks](https://mlsanity.com/paper/2202.03416/) (2022)
- [Filtered Noise Shaping for Time Domain Room Impulse Response Estimation From Reverberant Speech](https://mlsanity.com/paper/2107.07503/) (2021)
- [SkipConvNet: Skip Convolutional Neural Network for Speech Dereverberation using Optimally Smoothed Spectral Mapping](https://mlsanity.com/paper/2007.09131/) (2020)
- [TS-RIR: Translated synthetic room impulse responses for speech augmentation](https://mlsanity.com/paper/2103.16804/) (2021)
- [Improving Reverberant Speech Training Using Diffuse Acoustic Simulation](https://mlsanity.com/paper/1907.03988/) (2019)

## Built on by (most cited first, 23 of 23)

- [AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesis](https://mlsanity.com/paper/2302.02088/) (2023)
- [Novel-View Acoustic Synthesis](https://mlsanity.com/paper/2301.08730/) (2023)
- [Measuring Acoustics with Collaborative Multiple Agents](https://mlsanity.com/paper/2310.05368/) (2023)
- [AV-RIR: Audio-Visual Room Impulse Response Estimation](https://mlsanity.com/paper/2312.00834/) (2023)
- [Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmark](https://mlsanity.com/paper/2403.18821/) (2024)
- [AdVerb: Visually Guided Audio Dereverberation](https://mlsanity.com/paper/2308.12370/) (2023)
- [Sound propagation in realistic interactive 3D scenes with parameterized sources using deep neural operators](https://mlsanity.com/paper/2308.05141/) (2023)
- [Hearing Anything Anywhere](https://mlsanity.com/paper/2406.07532/) (2024)
- [Listen2Scene: Interactive material-aware binaural sound propagation for reconstructed 3D scenes](https://mlsanity.com/paper/2302.02809/) (2023)
- [Towards Improved Room Impulse Response Estimation for Speech Recognition](https://mlsanity.com/paper/2211.04473/) (2022)
- [Hearing Anywhere in Any Environment](https://mlsanity.com/paper/2504.10746/) (2025)
- [Novel-View Acoustic Synthesis from 3D Reconstructed Rooms](https://mlsanity.com/paper/2310.15130/) (2023)
- [Differentiable Room Acoustic Rendering with Multi-View Vision Priors](https://mlsanity.com/paper/2504.21847/) (2025)
- [SOAF: Scene Occlusion-aware Neural Acoustic Field](https://mlsanity.com/paper/2407.02264/) (2024)
- [ActiveRIR: Active Audio-Visual Exploration for Acoustic Environment Modeling](https://mlsanity.com/paper/2404.16216/) (2024)
- [Resounding Acoustic Fields with Reciprocity](https://mlsanity.com/paper/2510.20602/) (2025)
- [EM-GANSim: Real-time and Accurate EM Simulation Using Conditional GANs for 3D Indoor Scenes](https://mlsanity.com/paper/2405.17366/) (2024)
- [Explicit Context-Driven Neural Acoustic Modeling for High-Fidelity RIR Generation](https://mlsanity.com/paper/2509.15210/) (2025)
- [PromptReverb: Multimodal Room Impulse Response Generation Through Latent Rectified Flow Matching](https://mlsanity.com/paper/2510.22439/) (2025)
- [Reciprocal Latent Fields for Precomputed Sound Propagation](https://mlsanity.com/paper/2602.06937/) (2026)
- [Towards Improving Speaker Distance Estimation through Generative Impulse Response Augmentation](https://mlsanity.com/paper/2605.00721/) (2026)
- [Neural Radiated-Noise Fields for Unmanned Underwater Vehicle Noise Spectrum Prediction in Three-Dimensional Scenes](https://mlsanity.com/paper/2606.04008/) (2026)
- [PathRIR: Physics-Guided Acoustic Path Selection and Late-Tail Compensation for Fast Room Impulse Response Simulation](https://mlsanity.com/paper/2607.23293/) (2026)

## Abstract (arXiv)

We propose a mesh-based neural network (MESH2IR) to generate acoustic impulse responses (IRs) for indoor 3D scenes represented using a mesh. The IRs are used to create a high-quality sound experience in interactive applications and audio processing. Our method can handle input triangular meshes with arbitrary topologies (2K - 3M triangles). We present a novel training technique to train MESH2IR using energy decay relief and highlight its benefits. We also show that training MESH2IR on IRs preprocessed using our proposed technique significantly improves the accuracy of IR generation. We reduce the non-linearity in the mesh space by transforming 3D scene meshes to latent space using a graph convolution network. Our MESH2IR is more than 200 times faster than a geometric acoustic algorithm on a CPU and can generate more than 10,000 IRs per second on an NVIDIA GeForce RTX 2080 Ti GPU for a given furnished indoor 3D scene. The acoustic metrics are used to characterize the acoustic environment. We show that the acoustic metrics of the IRs predicted from our MESH2IR match the ground truth with less than 10% error. We also highlight the benefits of MESH2IR on audio and speech processing applications such as speech dereverberation and speech separation. To the best of our knowledge, ours is the first neural-network-based approach to predict IRs from a given 3D scene mesh in real-time.