# Investigating the Integration of Spatial Information in Foundation-Model-Based Speaker Diarization

arXiv: https://arxiv.org/abs/2607.12647 · HTML: https://mlsanity.com/paper/2607.12647/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 14 Jul 2026.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 20 of 20)

- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge](https://mlsanity.com/paper/2110.07393/) (2021)
- [AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario](https://mlsanity.com/paper/2104.03603/) (2021)
- [Integrating end-to-end neural and clustering-based diarization: Getting the best of both worlds](https://mlsanity.com/paper/2010.13366/) (2020)
- [End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation](https://mlsanity.com/paper/1910.14104/) (2019)
- [Advances in integration of end-to-end neural and clustering-based diarization for real conversational speech](https://mlsanity.com/paper/2105.09040/) (2021)
- [NOTSOFAR-1 Challenge: New Datasets, Baseline, and Tasks for Distant Meeting Transcription](https://mlsanity.com/paper/2401.08887/) (2024)
- [Leveraging Self-Supervised Learning for Speaker Diarization](https://mlsanity.com/paper/2409.09408/) (2024)
- [DOVER-Lap: A Method for Combining Overlap-aware Diarization Outputs](https://mlsanity.com/paper/2011.01997/) (2020)
- [DiPCo -- Dinner Party Corpus](https://mlsanity.com/paper/1909.13447/) (2019)
- [Recognizing Overlapped Speech in Meetings: A Multichannel Separation Approach Using Neural Networks](https://mlsanity.com/paper/1810.03655/) (2018)
- [Efficient and Robust Speaker Diarization via Structured Pruning of Self-Supervised Models](https://mlsanity.com/paper/2506.18623/) (2025)
- [UniX-Encoder: A Universal X-Channel Speech Encoder for Ad-Hoc Microphone Array Speech Processing](https://mlsanity.com/paper/2310.16367/) (2023)
- [Spatial-aware Speaker Diarization for Multi-channel Multi-party Meeting](https://mlsanity.com/paper/2209.12002/) (2022)
- [Concurrent Speaker Detection: A multi-microphone Transformer-Based Approach](https://mlsanity.com/paper/2403.06856/) (2024)
- [Spatio-spectral diarization of meetings by combining TDOA-based segmentation and speaker embedding-based clustering](https://mlsanity.com/paper/2506.16228/) (2025)
- [Spatially Aware Self-Supervised Models for Multi-Channel Neural Speaker Diarization](https://mlsanity.com/paper/2510.14551/) (2025)
- [On the Role of Spatial Features in Foundation-Model-Based Speaker Diarization](https://mlsanity.com/paper/2601.02231/) (2026)

## Abstract (arXiv)

Spatial information gleaned from multi-channel input has been shown to lead to improvements in meeting processing tasks like diarization and source separation. At the same time, diarization based on features extracted by large pretrained single-channel foundation models, such as WavLM, achieved state-of-the-art performance. This work compares three approaches to integrate spatial features into foundation model-based diarization systems: the cascade of a beamformer and a single-channel foundation model, a multi-channel foundation model, and the conditioning of the downstream network on explicitly extracted spatial features. Results show that the beamformer front-end is even detrimental to diarization performance in regions of overlapped speech, while best performance is achieved with the conditioning, demonstrating that the incorporation of explicit spatial features is a competitive approach to foundation-model-supported diarization. This approach is further subjected to a detailed error analysis showing that the conditioning system removes errors to a good extent that would occur when either only spectral or only spatial features were used.