# Measuring Acoustics with Collaborative Multiple Agents

arXiv: https://arxiv.org/abs/2310.05368 · HTML: https://mlsanity.com/paper/2310.05368/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 55 references that are in the map.
- First posted on arXiv 9 Oct 2023.

## Builds on (most cited first, 30 of 38)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.10800/) (2020)
- [Occupancy Anticipation for Efficient Exploration and Navigation](https://mlsanity.com/paper/2008.09285/) (2020)
- [Off-Policy Multi-Agent Decomposed Policy Gradients](https://mlsanity.com/paper/2007.12322/) (2020)
- [Look, Listen, and Act: Towards Audio-Visual Embodied Navigation](https://mlsanity.com/paper/1912.11684/) (2019)
- [Self-Supervised Generation of Spatial Audio for 360 Video](https://mlsanity.com/paper/1809.02587/) (2018)
- [Semantic Audio-Visual Navigation](https://mlsanity.com/paper/2012.11583/) (2020)
- [An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation](https://mlsanity.com/paper/2008.09586/) (2020)
- [Learning Neural Acoustic Fields](https://mlsanity.com/paper/2204.00628/) (2022)
- [VisualEchoes: Spatial Image Representation Learning through Echolocation](https://mlsanity.com/paper/2005.01616/) (2020)
- [Sound Adversarial Audio-Visual Navigation](https://mlsanity.com/paper/2202.10910/) (2022)
- [Few-Shot Audio-Visual Learning of Environment Acoustics](https://mlsanity.com/paper/2206.04006/) (2022)
- [gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Acceleration](https://mlsanity.com/paper/1810.11359/) (2018)
- [Visually Informed Binaural Audio Generation without Binaural Audios](https://mlsanity.com/paper/2104.06162/) (2021)
- [Visual Acoustic Matching](https://mlsanity.com/paper/2202.06875/) (2022)
- [See, Hear, Explore: Curiosity via Audio-Visual Association](https://mlsanity.com/paper/2007.03669/) (2020)
- [FAST-RIR: Fast neural diffuse room impulse response generator](https://mlsanity.com/paper/2110.04057/) (2021)
- [Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis](https://mlsanity.com/paper/2103.14201/) (2021)
- [Audio-Visual Floorplan Reconstruction](https://mlsanity.com/paper/2012.15470/) (2020)
- [Egocentric Deep Multi-Channel Audio-Visual Active Speaker Localization](https://mlsanity.com/paper/2201.01928/) (2022)
- [BatVision: Learning to See 3D Spatial Layout with Two Ears](https://mlsanity.com/paper/1912.07011/) (2019)
- [GWA: A Large High-Quality Acoustic Dataset for Audio Processing](https://mlsanity.com/paper/2204.01787/) (2022)

## Built on by (most cited first, 22 of 22)

- [Audio-Guided Dynamic Modality Fusion with Stereo-Aware Attention for Audio-Visual Navigation](https://mlsanity.com/paper/2509.16924/) (2025)
- [Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environments](https://mlsanity.com/paper/2509.22698/) (2025)
- [Iterative Residual Cross-Attention Mechanism: An Integrated Approach for Audio-Visual Navigation Tasks](https://mlsanity.com/paper/2509.25652/) (2025)
- [Dynamic Multi-Target Fusion for Efficient Audio-Visual Navigation](https://mlsanity.com/paper/2509.21377/) (2025)
- [Modality-Invariant Bidirectional Temporal Representation Distillation Network for Missing Multimodal Sentiment Analysis](https://mlsanity.com/paper/2501.05474/) (2025)
- [ActiveRIR: Active Audio-Visual Exploration for Acoustic Environment Modeling](https://mlsanity.com/paper/2404.16216/) (2024)
- [EGSTalker: Real-Time Audio-Driven Talking Head Generation with Efficient Gaussian Deformation](https://mlsanity.com/paper/2510.08587/) (2025)
- [Audio-Guided Visual Perception for Audio-Visual Navigation](https://mlsanity.com/paper/2510.11760/) (2025)
- [Residual Cross-Modal Fusion Networks for Audio-Visual Navigation](https://mlsanity.com/paper/2601.08868/) (2026)
- [Landmark-Guided Knowledge for Vision-and-Language Navigation](https://mlsanity.com/paper/2509.25655/) (2025)
- [Audio Spatially-Guided Fusion for Audio-Visual Navigation](https://mlsanity.com/paper/2604.02389/) (2026)
- [Spatial-Aware Conditioned Fusion for Audio-Visual Navigation](https://mlsanity.com/paper/2604.02390/) (2026)
- [Reliability-Aware Geometric Fusion for Robust Audio-Visual Navigation](https://mlsanity.com/paper/2604.02391/) (2026)
- [Generalizable Audio-Visual Navigation via Binaural Difference Attention and Action Transition Prediction](https://mlsanity.com/paper/2604.05007/) (2026)
- [EAD-Net: Emotion-Aware Talking Head Generation with Spatial Refinement and Temporal Coherence](https://mlsanity.com/paper/2604.23325/) (2026)
- [MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysis](https://mlsanity.com/paper/2607.10599/) (2026)
- [HRO: Hierarchical Room-to-Object Framework for Zero-Shot Object Goal Navigation with Large Language Models](https://mlsanity.com/paper/2607.13072/) (2026)
- [CTAN: Cycle-Temporal Attention Network for Embodied Audio-Visual Navigation](https://mlsanity.com/paper/2609.17420/) (2026)
- [Transformer-Based Token Fusion and Dynamic Graph Planning for Audio-Visual Navigation](https://mlsanity.com/paper/2609.17421/) (2026)
- [Talking Head Synthesis with Facial Landmark Guidance via 3D Gaussian Splatting](https://mlsanity.com/paper/2609.17422/) (2026)
- [WS-NeRF: A Mamba-Driven World-State-Aware Adaptive Deblurring Neural Radiance Field](https://mlsanity.com/paper/2609.21391/) (2026)
- [Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregation](https://mlsanity.com/paper/2610.14077/) (2026)

## Abstract (arXiv)

As humans, we hear sound every second of our life. The sound we hear is often affected by the acoustics of the environment surrounding us. For example, a spacious hall leads to more reverberation. Room Impulse Responses (RIR) are commonly used to characterize environment acoustics as a function of the scene geometry, materials, and source/receiver locations. Traditionally, RIRs are measured by setting up a loudspeaker and microphone in the environment for all source/receiver locations, which is time-consuming and inefficient. We propose to let two robots measure the environment's acoustics by actively moving and emitting/receiving sweep signals. We also devise a collaborative multi-agent policy where these two robots are trained to explore the environment's acoustics while being rewarded for wide exploration and accurate prediction. We show that the robots learn to collaborate and move to explore environment acoustics while minimizing the prediction error. To the best of our knowledge, we present the very first problem formulation and solution to the task of collaborative environment acoustics measurements with multiple agents.