mlsanityData to 14 Oct 2026

Auditory Intelligence: Understanding the World Through Sound

arXiv 2508.07829 ↗First posted 11 Aug 2025eess.AS cs.AI cs.SDcited by 1

Not on the map. Why?

Abstract (arXiv)

Recent progress in auditory intelligence has yielded high-performing systems for sound event detection (SED), acoustic scene classification (ASC), automated audio captioning (AAC), and audio question answering (AQA). Yet these tasks remain largely constrained to surface-level recognition-capturing what happened but not why, what it implies, or how it unfolds in context. I propose a conceptual reframing of auditory intelligence as a layered, situated process that encompasses perception, reasoning, and interaction. To instantiate this view, I introduce four cognitively inspired task paradigms-ASPIRE, SODA, AUX, and AUGMENT-those structure auditory understanding across time-frequency pattern captioning, hierarchical event/scene description, causal explanation, and goal-driven interpretation, respectively. Together, these paradigms provide a roadmap toward more generalizable, explainable, and human-aligned auditory intelligence, and are intended to catalyze a broader discussion of what it means for machines to understand sound.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 39 of its 52 references that are in the map. First posted on arXiv 11 Aug 2025.

Builds on

39 of 52 references.

References · 39

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  7. 2021AST: Audio Spectrogram Transformercited by 643
  8. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 599
  9. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
  10. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 388
  11. 2019Clotho: An Audio Captioning Datasetcited by 358
  12. 2022AudioGen: Textually Guided Audio Generationcited by 295
27 more
  1. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 267
  2. 2018Attentive Statistics Pooling for Deep Speaker Embeddingcited by 114
  3. 2018Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition Systemcited by 47
  4. 2017Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detectioncited by 46
  5. 2017Automated Audio Captioning with Recurrent Neural Networkscited by 40
  6. 2020Overview and Evaluation of Sound Event Localization and Detection in DCASE 2019cited by 31
  7. 2019A Framework for the Robust Evaluation of Sound Event Detectioncited by 29
  8. 2022Frequency Dynamic Convolution: Frequency-Adaptive Pattern Recognition for Sound Event Detectioncited by 23
  9. 2025FLAM: Frame-Wise Language-Audio Modelingcited by 18
  10. 2022STARSS22: A dataset of spatial recordings of real scenes with spatiotemporal annotations of sound eventscited by 16
  11. 2021Integrating Frequency Translational Invariance in TDNNs and Frequency Positional Information in 2D ResNets to Enhance Speaker Verificationcited by 11
  12. 2021FilterAugment: An Acoustic Environmental Data Augmentation Methodcited by 8
  13. 2024Mind the Domain Gap: a Systematic Analysis on Bioacoustic Sound Event Detectioncited by 7
  14. 2021Heavily Augmented Sound Event Detection utilizing Weak Predictionscited by 6
  15. 2025Binaural Sound Event Localization and Detection based on HRTF Cues for Humanoid Robotscited by 3
  16. 2021Deep learning based cough detection camera using enhanced featurescited by 2
  17. 2022Convolution-Based Channel-Frequency Attention for Text-Independent Speaker Verificationcited by 2
  18. 2023VIFS: An End-to-End Variational Inference for Foley Sound Synthesiscited by 2
  19. 2024Diversifying and Expanding Frequency-Adaptive Convolution Kernels for Sound Event Detectioncited by 2
  20. 2024Pushing the Limit of Sound Event Detection with Multi-Dilated Frequency Dynamic Convolutioncited by 2
  21. 2024Self Training and Ensembling Frequency Dependent Networks with Coarse Prediction Pooling and Sound Event Bounding Boxescited by 2
  22. 2025JiTTER: Jigsaw Temporal Transformer for Event Reconstruction for Self-Supervised Sound Event Detectioncited by 2
  23. 2025Temporal Attention Pooling for Frequency Dynamic Convolution in Sound Event Detectioncited by 2
  24. 2022Data Augmentation and Squeeze-and-Excitation Network on Multiple Dimension for Sound Event Localization and Detection in Real Scenescited by 1
  25. 2025Towards Understanding of Frequency Dependence on Sound Event Detectioncited by 1
  26. 2025DNN based HRIRs Identification with a Continuously Rotating Speaker Arraycited by 1
  27. 2025Frequency Dynamic Convolutions for Sound Event Detectioncited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025CMDAR: A Chinese Multi-scene Dynamic Audio Reasoning Benchmark with Diverse Challengescited by 2