mlsanityData to 14 Oct 2026

Measuring Acoustics with Collaborative Multiple Agents

arXiv 2310.05368 ↗First posted 9 Oct 2023cs.AI cs.MA cs.SDcited by 22

Not on the map. Why?

Abstract (arXiv)

As humans, we hear sound every second of our life. The sound we hear is often affected by the acoustics of the environment surrounding us. For example, a spacious hall leads to more reverberation. Room Impulse Responses (RIR) are commonly used to characterize environment acoustics as a function of the scene geometry, materials, and source/receiver locations. Traditionally, RIRs are measured by setting up a loudspeaker and microphone in the environment for all source/receiver locations, which is time-consuming and inefficient. We propose to let two robots measure the environment's acoustics by actively moving and emitting/receiving sweep signals. We also devise a collaborative multi-agent policy where these two robots are trained to explore the environment's acoustics while being rewarded for wide exploration and accurate prediction. We show that the robots learn to collaborate and move to explore environment acoustics while minimizing the prediction error. To the best of our knowledge, we present the very first problem formulation and solution to the task of collaborative environment acoustics measurements with multiple agents.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 38 of its 55 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

38 of 55 references.

References · 38

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  6. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  7. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  8. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  9. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  10. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  11. 2020Occupancy Anticipation for Efficient Exploration and Navigationcited by 106
  12. 2020Off-Policy Multi-Agent Decomposed Policy Gradientscited by 98
26 more
  1. 2019Look, Listen, and Act: Towards Audio-Visual Embodied Navigationcited by 96
  2. 2018Self-Supervised Generation of Spatial Audio for 360 Videocited by 93
  3. 2020Semantic Audio-Visual Navigationcited by 82
  4. 2020An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separationcited by 81
  5. 2022Learning Neural Acoustic Fieldscited by 65
  6. 2020VisualEchoes: Spatial Image Representation Learning through Echolocationcited by 55
  7. 2022Sound Adversarial Audio-Visual Navigationcited by 48
  8. 2022Few-Shot Audio-Visual Learning of Environment Acousticscited by 48
  9. 2018gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Accelerationcited by 46
  10. 2021Visually Informed Binaural Audio Generation without Binaural Audioscited by 46
  11. 2022Visual Acoustic Matchingcited by 45
  12. 2020See, Hear, Explore: Curiosity via Audio-Visual Associationcited by 43
  13. 2021FAST-RIR: Fast neural diffuse room impulse response generatorcited by 35
  14. 2021Image2Reverb: Cross-Modal Reverb Impulse Response Synthesiscited by 34
  15. 2020Audio-Visual Floorplan Reconstructioncited by 32
  16. 2022Egocentric Deep Multi-Channel Audio-Visual Active Speaker Localizationcited by 27
  17. 2019BatVision: Learning to See 3D Spatial Layout with Two Earscited by 26
  18. 2022GWA: A Large High-Quality Acoustic Dataset for Audio Processingcited by 25
  19. 2022MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenescited by 23
  20. 2021Geometry-Aware Multi-Task Learning for Binaural Audio Generation from Videocited by 22
  21. 2022Pay Self-Attention to Audio-Visual Navigationcited by 18
  22. 2022Active Audio-Visual Separation of Dynamic Sound Sourcescited by 16
  23. 2022Finding Fallen Objects Via Asynchronous Audio-Visual Integrationcited by 14
  24. 2019Improving Reverberant Speech Training Using Diffuse Acoustic Simulationcited by 9
  25. 2021Multi-Channel Multi-Speaker ASR Using 3D Spatial Featurecited by 7
  26. 2020StoRIR: Stochastic Room Impulse Response Generation for Audio Data Augmentationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 13.

22 papers.

Sort:

By area:

  1. 2025Audio-Guided Dynamic Modality Fusion with Stereo-Aware Attention for Audio-Visual Navigationcited by 15
  2. 2025Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environmentscited by 14
  3. 2025Iterative Residual Cross-Attention Mechanism: An Integrated Approach for Audio-Visual Navigation Taskscited by 14
  4. 2025Dynamic Multi-Target Fusion for Efficient Audio-Visual Navigationcited by 13
  5. 2025Modality-Invariant Bidirectional Temporal Representation Distillation Network for Missing Multimodal Sentiment Analysiscited by 8
  6. 2024ActiveRIR: Active Audio-Visual Exploration for Acoustic Environment Modelingcited by 2
  7. 2025EGSTalker: Real-Time Audio-Driven Talking Head Generation with Efficient Gaussian Deformationcited by 1
  8. 2025Audio-Guided Visual Perception for Audio-Visual Navigationcited by 1
  9. 2026Residual Cross-Modal Fusion Networks for Audio-Visual Navigationcited by 1
  10. 2025Landmark-Guided Knowledge for Vision-and-Language Navigationcited by 0
  11. 2026Audio Spatially-Guided Fusion for Audio-Visual Navigationcited by 0
  12. 2026Spatial-Aware Conditioned Fusion for Audio-Visual Navigationcited by 0
10 more
  1. 2026Reliability-Aware Geometric Fusion for Robust Audio-Visual Navigationcited by 0
  2. 2026Generalizable Audio-Visual Navigation via Binaural Difference Attention and Action Transition Predictioncited by 0
  3. 2026EAD-Net: Emotion-Aware Talking Head Generation with Spatial Refinement and Temporal Coherencecited by 0
  4. 2026MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysiscited by 0
  5. 2026HRO: Hierarchical Room-to-Object Framework for Zero-Shot Object Goal Navigation with Large Language Modelscited by 0
  6. 2026CTAN: Cycle-Temporal Attention Network for Embodied Audio-Visual Navigationcited by 0
  7. 2026Transformer-Based Token Fusion and Dynamic Graph Planning for Audio-Visual Navigationcited by 0
  8. 2026Talking Head Synthesis with Facial Landmark Guidance via 3D Gaussian Splattingcited by 0
  9. 2026WS-NeRF: A Mamba-Driven World-State-Aware Adaptive Deblurring Neural Radiance Fieldcited by 0
  10. 2026Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregationcited by 0