# Multi-agent Auditory Scene Analysis

arXiv: https://arxiv.org/abs/2507.02755 · HTML: https://mlsanity.com/paper/2507.02755/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 69 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks](https://mlsanity.com/paper/1701.05923/) (2017)
- [Robot Operating System 2: Design, Architecture, and Uses In The Wild](https://mlsanity.com/paper/2211.07752/) (2022)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [TorchAudio-Squim: Reference-less Speech Quality and Intelligibility measures in TorchAudio](https://mlsanity.com/paper/2304.01448/) (2023)
- [SCOREQ: Speech Quality Assessment with Contrastive Regression](https://mlsanity.com/paper/2410.06675/) (2024)
- [Learnable Graph Convolutional Network and Feature Fusion for Multi-view Learning](https://mlsanity.com/paper/2211.09155/) (2022)
- [Lightweight and Optimized Sound Source Localization and Tracking Methods for Open and Closed Microphone Array Configurations](https://mlsanity.com/paper/1812.00115/) (2018)
- [ODAS: Open embeddeD Audition System](https://mlsanity.com/paper/2103.03954/) (2021)
- [A ROS2 based communication architecture for control in collaborative and intelligent automation systems](https://mlsanity.com/paper/1905.09654/) (2019)
- [Direction of Arrival Correction through Speech Quality Feedback](https://mlsanity.com/paper/2408.07234/) (2024)
- [Lightweight Speaker Verification for Online Identification of New Speakers with Short Segments](https://mlsanity.com/paper/2003.03432/) (2020)

## Abstract (arXiv)

Auditory scene analysis (ASA) aims to retrieve information from the acoustic environment, by carrying out three main tasks: sound source location, separation, and classification. These tasks are traditionally executed with a linear data flow, where the sound sources are first located; then, using their location, each source is separated into its own audio stream; from each of which, information is extracted that is relevant to the application scenario (audio event detection, speaker identification, emotion classification, etc.). However, running these tasks linearly increases the overall response time, while making the last tasks (separation and classification) highly sensitive to errors of the first task (location). A considerable amount of effort and computational complexity has been employed in the state-of-the-art to develop techniques that are the least error-prone possible. However, doing so gives rise to an ASA system that is non-viable in many applications that require a small computational footprint and a low response time, such as bioacoustics, hearing-aid design, search and rescue, human-robot interaction, etc. To this effect, in this work, a multi-agent approach is proposed to carry out ASA where the tasks are run in parallel, with feedback loops between them to compensate for local errors, such as: using the quality of the separation output to correct the location error; and using the classification result to reduce the localization's sensitivity towards interferences. The result is a multi-agent auditory scene analysis (MASA) system that is robust against local errors, without a considerable increase in complexity, and with a low response time. The complete proposed MASA system is provided as a publicly available framework that uses open-source tools for sound acquisition and reproduction (JACK) and inter-agent communication (ROS2), allowing users to add their own agents.