mlsanityData to 19 Oct 2026

Spatial Speech Perception Systems: A Survey of Sound Source Localization, Directional Enhancement, and Speech Recognition

arXiv 2607.02296 ↗First posted 2 Jul 2026eess.AScited by 0

On the map: Speech and audio language models → Speech enhancement and noise suppression

Abstract (arXiv)

Robust speech understanding in real-world acoustic environments remains a fundamental challenge for intelligent auditory systems such as robot audition, hearing aids, teleconferencing systems, smart speakers, and voice-controlled assistants. These systems must operate under background noise, reverberation, competing speakers, and dynamic acoustic conditions. Spatial speech perception addresses this challenge by exploiting microphone-array information to localize, enhance, and interpret target speech in complex acoustic scenes. This paper surveys spatial speech perception systems with emphasis on the roles of sound source localization (SSL), directional speech enhancement (DSE), and automatic speech recognition (ASR), both individually and within integrated processing pipelines. We review classical signal-processing approaches and recent learning-based methods for microphone-array localization, beamforming, neural enhancement, speech separation, and modern recognition architectures. Beyond component-level analysis, we discuss robustness to noise and reverberation, multi-speaker operation, real-time constraints, and computational efficiency. We also examine representative applications in robot audition, hearing assistance, smart speakers, and teleconferencing, and identify open challenges and future directions toward robust, low-latency, and perception-aware speech systems for complex acoustic environments.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models. Builds on 49 of its 108 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

49 of 108 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Neural Directed Speech Enhancement with Dual Microphone Array in High Noise Scenario24 Dec
  2. 2022Insights Into Deep Non-linear Filters for Improved Multi-channel Speech Enhancement27 Jun

Other references · 47

  1. 2017Attention Is All You Needcited by 61,180
  2. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,870
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,661
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 4,385
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 2,293
  6. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 774
  7. 2015Listen, Attend and Spellcited by 635
  8. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 538
  9. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 497
  10. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 383
  11. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 281
  12. 2020CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordingscited by 231
35 more
  1. 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 178
  2. 2018The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselinescited by 154
  3. 2020Continuous speech separation: dataset and analysiscited by 124
  4. 2018Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducercited by 122
  5. 2019Look, Listen, and Act: Towards Audio-Visual Embodied Navigationcited by 98
  6. 2018Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identificationcited by 97
  7. 2018Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networkscited by 90
  8. 2020FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularizationcited by 82
  9. 2019FaSNet: Low-latency Adaptive Beamforming for Multi-microphone Audio Processingcited by 52
  10. 2020Far-Field Automatic Speech Recognitioncited by 50
  11. 2021A Survey of Sound Source Localization with Deep Learning Methodscited by 50
  12. 2016Deep attractor network for single-microphone speaker separationcited by 40
  13. 2021Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognitioncited by 29
  14. 2023Turning Whisper into Real-Time Transcription Systemcited by 29
  15. 2017Deep Neural Networks for Multiple Speaker Detection and Localizationcited by 25
  16. 2017Broadband DOA estimation using Convolutional neural networks trained with noise signalscited by 19
  17. 2020Directional ASR: A New Paradigm for E2E Multi-Speaker Speech Recognition with Source Localizationcited by 11
  18. 2024All Neural Low-latency Directional Speech Extractioncited by 9
  19. 2025Simultaneous Translation with Offline Speech and LLM Models in CUNI Submission to IWSLT 2025cited by 9
  20. 2023Self-Supervised Adaptive AV Fusion Module for Pre-Trained ASR Modelscited by 8
  21. 2021ODAS: Open embeddeD Audition Systemcited by 6
  22. 2024Conformer-1: Robust ASR via Large-Scale Semisupervised Bootstrappingcited by 6
  23. 2025End-to-End DOA-Guided Speech Extraction in Noisy Multi-Talker Scenarioscited by 3
  24. 2025Sound Source Localization for Human-Robot Interaction in Outdoor Environmentscited by 3
  25. 2019Fast and Robust 3-D Sound Source Localization with DSVD-PHATcited by 1
  26. 2019Cooperative Audio Source Separation and Enhancement Using Distributed Microphone Arrays and Wearable Devicescited by 1
  27. 2020Binaural Audio Source Remixing with Microphone Array Listening Devicescited by 1
  28. 2023Ego-noise reduction of a mobile robot using noise spatial covariance matrix learning and minimum variance distortionless responsecited by 1
  29. 2024Real-time Stereo Speech Enhancement with Spatial-Cue Preservation based on Dual-Path Structurecited by 1
  30. 2024Syllable based DNN-HMM Cantonese Speech to Text Systemcited by 1
  31. 2025AuralNet: Hierarchical Attention-based 3D Binaural Localization of Overlapping Speakerscited by 1
  32. 2025Single-Microphone-Based Sound Source Localization for Mobile Robots in Reverberant Environmentscited by 1
  33. 2025Multi-Speaker DOA Estimation in Binaural Hearing Aids using Deep Learning and Speaker Count Fusioncited by 1
  34. 2025IPDnet2: an efficient and improved inter-channel phase difference estimation network for sound source localizationcited by 1
  35. 2026Lend me an Ear: Speech Enhancement Using a Robotic Arm with a Microphone Arraycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MiDashengLM-Spatial: Unifying General Audio Understanding and Spatial Awareness
  2. 2026SEA-LM: Egocentric Spatial Audio Understanding for Wearable Microphone Arrays
  3. 2024Can Large Language Models Understand Spatial Audio?
  4. 2026SpatialEmb: Extract and Encode Spatial Information for 1-Stage Multi-channel Multi-speaker ASR on Arbitrary Microphone Arrays
  5. 2025SPUR: A Plug-and-Play Framework for Integrating Spatial Audio Understanding and Reasoning into Large Audio-Language Models
  6. 2024BAT: Learning to Reason about Spatial Sounds with Large Language Models
  7. 2023Localizing Spatial Information in Neural Spatiospectral Filters
  8. 2025Sci-Phi: A Large Language Model Spatial Audio Descriptor