Measuring Acoustics with Collaborative Multiple Agents
Not on the map. Why?
Abstract (arXiv)
As humans, we hear sound every second of our life. The sound we hear is often affected by the acoustics of the environment surrounding us. For example, a spacious hall leads to more reverberation. Room Impulse Responses (RIR) are commonly used to characterize environment acoustics as a function of the scene geometry, materials, and source/receiver locations. Traditionally, RIRs are measured by setting up a loudspeaker and microphone in the environment for all source/receiver locations, which is time-consuming and inefficient. We propose to let two robots measure the environment's acoustics by actively moving and emitting/receiving sweep signals. We also devise a collaborative multi-agent policy where these two robots are trained to explore the environment's acoustics while being rewarded for wide exploration and accurate prediction. We show that the robots learn to collaborate and move to explore environment acoustics while minimizing the prediction error. To the best of our knowledge, we present the very first problem formulation and solution to the task of collaborative environment acoustics measurements with multiple agents.
Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 38 of its 55 references that are in the map. First posted on arXiv 9 Oct 2023.
Builds on
38 of 55 references.
References · 38
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
- 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
- 2017Counterfactual Multi-Agent Policy Gradientscited by 751
- 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
- 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
- 2020Occupancy Anticipation for Efficient Exploration and Navigationcited by 106
- 2020Off-Policy Multi-Agent Decomposed Policy Gradientscited by 98
26 more
- 2019Look, Listen, and Act: Towards Audio-Visual Embodied Navigationcited by 96
- 2018Self-Supervised Generation of Spatial Audio for 360 Videocited by 93
- 2020Semantic Audio-Visual Navigationcited by 82
- 2020An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separationcited by 81
- 2022Learning Neural Acoustic Fieldscited by 65
- 2020VisualEchoes: Spatial Image Representation Learning through Echolocationcited by 55
- 2022Sound Adversarial Audio-Visual Navigationcited by 48
- 2022Few-Shot Audio-Visual Learning of Environment Acousticscited by 48
- 2018gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Accelerationcited by 46
- 2021Visually Informed Binaural Audio Generation without Binaural Audioscited by 46
- 2022Visual Acoustic Matchingcited by 45
- 2020See, Hear, Explore: Curiosity via Audio-Visual Associationcited by 43
- 2021FAST-RIR: Fast neural diffuse room impulse response generatorcited by 35
- 2021Image2Reverb: Cross-Modal Reverb Impulse Response Synthesiscited by 34
- 2020Audio-Visual Floorplan Reconstructioncited by 32
- 2022Egocentric Deep Multi-Channel Audio-Visual Active Speaker Localizationcited by 27
- 2019BatVision: Learning to See 3D Spatial Layout with Two Earscited by 26
- 2022GWA: A Large High-Quality Acoustic Dataset for Audio Processingcited by 25
- 2022MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenescited by 23
- 2021Geometry-Aware Multi-Task Learning for Binaural Audio Generation from Videocited by 22
- 2022Pay Self-Attention to Audio-Visual Navigationcited by 18
- 2022Active Audio-Visual Separation of Dynamic Sound Sourcescited by 16
- 2022Finding Fallen Objects Via Asynchronous Audio-Visual Integrationcited by 14
- 2019Improving Reverberant Speech Training Using Diffuse Acoustic Simulationcited by 9
- 2021Multi-Channel Multi-Speaker ASR Using 3D Spatial Featurecited by 7
- 2020StoRIR: Stochastic Room Impulse Response Generation for Audio Data Augmentationcited by 2
Built on by
Citing papers per year: 2024: 1; 2025: 8; 2026: 13.
22 papers.
Sort:
By area:
- 2025Audio-Guided Dynamic Modality Fusion with Stereo-Aware Attention for Audio-Visual Navigationcited by 15
- 2025Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environmentscited by 14
- 2025Iterative Residual Cross-Attention Mechanism: An Integrated Approach for Audio-Visual Navigation Taskscited by 14
- 2025Dynamic Multi-Target Fusion for Efficient Audio-Visual Navigationcited by 13
- 2025Modality-Invariant Bidirectional Temporal Representation Distillation Network for Missing Multimodal Sentiment Analysiscited by 8
- 2024ActiveRIR: Active Audio-Visual Exploration for Acoustic Environment Modelingcited by 2
- 2025EGSTalker: Real-Time Audio-Driven Talking Head Generation with Efficient Gaussian Deformationcited by 1
- 2025Audio-Guided Visual Perception for Audio-Visual Navigationcited by 1
- 2026Residual Cross-Modal Fusion Networks for Audio-Visual Navigationcited by 1
- 2025Landmark-Guided Knowledge for Vision-and-Language Navigationcited by 0
- 2026Audio Spatially-Guided Fusion for Audio-Visual Navigationcited by 0
- 2026Spatial-Aware Conditioned Fusion for Audio-Visual Navigationcited by 0
10 more
- 2026Reliability-Aware Geometric Fusion for Robust Audio-Visual Navigationcited by 0
- 2026Generalizable Audio-Visual Navigation via Binaural Difference Attention and Action Transition Predictioncited by 0
- 2026EAD-Net: Emotion-Aware Talking Head Generation with Spatial Refinement and Temporal Coherencecited by 0
- 2026MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysiscited by 0
- 2026HRO: Hierarchical Room-to-Object Framework for Zero-Shot Object Goal Navigation with Large Language Modelscited by 0
- 2026CTAN: Cycle-Temporal Attention Network for Embodied Audio-Visual Navigationcited by 0
- 2026Transformer-Based Token Fusion and Dynamic Graph Planning for Audio-Visual Navigationcited by 0
- 2026Talking Head Synthesis with Facial Landmark Guidance via 3D Gaussian Splattingcited by 0
- 2026WS-NeRF: A Mamba-Driven World-State-Aware Adaptive Deblurring Neural Radiance Fieldcited by 0
- 2026Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregationcited by 0
- 2026Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregation8 Oct 2026
- 2026WS-NeRF: A Mamba-Driven World-State-Aware Adaptive Deblurring Neural Radiance Field18 Sep 2026
- 2026CTAN: Cycle-Temporal Attention Network for Embodied Audio-Visual Navigation25 Jul 2026
- 2026Talking Head Synthesis with Facial Landmark Guidance via 3D Gaussian Splatting16 Jul 2026
- 2026Transformer-Based Token Fusion and Dynamic Graph Planning for Audio-Visual Navigation16 Jul 2026
- 2026HRO: Hierarchical Room-to-Object Framework for Zero-Shot Object Goal Navigation with Large Language Models12 Jul 2026
- 2026MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysis12 Jul 2026
- 2026EAD-Net: Emotion-Aware Talking Head Generation with Spatial Refinement and Temporal Coherence25 Apr 2026
- 2026Generalizable Audio-Visual Navigation via Binaural Difference Attention and Action Transition Prediction6 Apr 2026
- 2026Reliability-Aware Geometric Fusion for Robust Audio-Visual Navigation2 Apr 2026
- 2026Spatial-Aware Conditioned Fusion for Audio-Visual Navigation2 Apr 2026
- 2026Audio Spatially-Guided Fusion for Audio-Visual Navigation2 Apr 2026
10 more
- 2026Residual Cross-Modal Fusion Networks for Audio-Visual Navigation11 Jan 2026
- 2025Audio-Guided Visual Perception for Audio-Visual Navigation13 Oct 2025
- 2025EGSTalker: Real-Time Audio-Driven Talking Head Generation with Efficient Gaussian Deformation3 Oct 2025
- 2025Landmark-Guided Knowledge for Vision-and-Language Navigation30 Sep 2025
- 2025Iterative Residual Cross-Attention Mechanism: An Integrated Approach for Audio-Visual Navigation Tasks30 Sep 2025
- 2025Dynamic Multi-Target Fusion for Efficient Audio-Visual Navigation23 Sep 2025
- 2025Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environments21 Sep 2025
- 2025Audio-Guided Dynamic Modality Fusion with Stereo-Aware Attention for Audio-Visual Navigation21 Sep 2025
- 2025Modality-Invariant Bidirectional Temporal Representation Distillation Network for Missing Multimodal Sentiment Analysis7 Jan 2025
- 2024ActiveRIR: Active Audio-Visual Exploration for Acoustic Environment Modeling24 Apr 2024
- 2025Audio-Guided Dynamic Modality Fusion with Stereo-Aware Attention for Audio-Visual Navigationcited by 15
- 2025Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environmentscited by 14
- 2025Iterative Residual Cross-Attention Mechanism: An Integrated Approach for Audio-Visual Navigation Taskscited by 14
- 2025Dynamic Multi-Target Fusion for Efficient Audio-Visual Navigationcited by 13
- 2025Modality-Invariant Bidirectional Temporal Representation Distillation Network for Missing Multimodal Sentiment Analysiscited by 8
- 2024ActiveRIR: Active Audio-Visual Exploration for Acoustic Environment Modelingcited by 2
- 2025EGSTalker: Real-Time Audio-Driven Talking Head Generation with Efficient Gaussian Deformationcited by 1
- 2025Audio-Guided Visual Perception for Audio-Visual Navigationcited by 1
- 2026Residual Cross-Modal Fusion Networks for Audio-Visual Navigationcited by 1
- 2025Landmark-Guided Knowledge for Vision-and-Language Navigationcited by 0
- 2026Audio Spatially-Guided Fusion for Audio-Visual Navigationcited by 0
- 2026Spatial-Aware Conditioned Fusion for Audio-Visual Navigationcited by 0
12 most cited of 21 All 21 (JSON)
- 2026Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregation8 Oct 2026
- 2026CTAN: Cycle-Temporal Attention Network for Embodied Audio-Visual Navigation25 Jul 2026
- 2026Talking Head Synthesis with Facial Landmark Guidance via 3D Gaussian Splatting16 Jul 2026
- 2026Transformer-Based Token Fusion and Dynamic Graph Planning for Audio-Visual Navigation16 Jul 2026
- 2026HRO: Hierarchical Room-to-Object Framework for Zero-Shot Object Goal Navigation with Large Language Models12 Jul 2026
- 2026MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysis12 Jul 2026
- 2026EAD-Net: Emotion-Aware Talking Head Generation with Spatial Refinement and Temporal Coherence25 Apr 2026
- 2026Generalizable Audio-Visual Navigation via Binaural Difference Attention and Action Transition Prediction6 Apr 2026
- 2026Reliability-Aware Geometric Fusion for Robust Audio-Visual Navigation2 Apr 2026
- 2026Spatial-Aware Conditioned Fusion for Audio-Visual Navigation2 Apr 2026
- 2026Audio Spatially-Guided Fusion for Audio-Visual Navigation2 Apr 2026
- 2026Residual Cross-Modal Fusion Networks for Audio-Visual Navigation11 Jan 2026
12 newest of 21 All 21 (JSON)