# Spatially Aware Self-Supervised Models for Multi-Channel Neural Speaker Diarization

arXiv: https://arxiv.org/abs/2510.14551 · HTML: https://mlsanity.com/paper/2510.14551/

## Key facts

- Cited by 3 papers in the map (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings](https://mlsanity.com/paper/2004.09249/) (2020)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge](https://mlsanity.com/paper/2110.07393/) (2021)
- [Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks](https://mlsanity.com/paper/2012.14952/) (2020)
- [AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario](https://mlsanity.com/paper/2104.03603/) (2021)
- [Integrating end-to-end neural and clustering-based diarization: Getting the best of both worlds](https://mlsanity.com/paper/2010.13366/) (2020)
- [End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation](https://mlsanity.com/paper/1910.14104/) (2019)
- [Advances in integration of end-to-end neural and clustering-based diarization for real conversational speech](https://mlsanity.com/paper/2105.09040/) (2021)
- [NOTSOFAR-1 Challenge: New Datasets, Baseline, and Tasks for Distant Meeting Transcription](https://mlsanity.com/paper/2401.08887/) (2024)
- [Leveraging Self-Supervised Learning for Speaker Diarization](https://mlsanity.com/paper/2409.09408/) (2024)
- [DOVER-Lap: A Method for Combining Overlap-aware Diarization Outputs](https://mlsanity.com/paper/2011.01997/) (2020)
- [Fine-tune Before Structured Pruning: Towards Compact and Accurate Self-Supervised Models for Speaker Diarization](https://mlsanity.com/paper/2505.24111/) (2025)
- [Cross-Channel Attention-Based Target Speaker Voice Activity Detection: Experimental Results for M2MeT Challenge](https://mlsanity.com/paper/2202.02687/) (2022)
- [Efficient and Robust Speaker Diarization via Structured Pruning of Self-Supervised Models](https://mlsanity.com/paper/2506.18623/) (2025)
- [Multi-Channel End-to-End Neural Diarization with Distributed Microphones](https://mlsanity.com/paper/2110.04694/) (2021)
- [The USTC-NERCSLIP Systems for the CHiME-7 DASR Challenge](https://mlsanity.com/paper/2308.14638/) (2023)
- [NTT speaker diarization system for CHiME-7: multi-domain, multi-microphone End-to-end and vector clustering diarization](https://mlsanity.com/paper/2309.12656/) (2023)
- [Semi-supervised multi-channel speaker diarization with cross-channel attention](https://mlsanity.com/paper/2307.08688/) (2023)
- [Spatio-spectral diarization of meetings by combining TDOA-based segmentation and speaker embedding-based clustering](https://mlsanity.com/paper/2506.16228/) (2025)

## Built on by (most cited first, 3 of 3)

- [On the Role of Spatial Features in Foundation-Model-Based Speaker Diarization](https://mlsanity.com/paper/2601.02231/) (2026)
- [Investigating the Integration of Spatial Information in Foundation-Model-Based Speaker Diarization](https://mlsanity.com/paper/2607.12647/) (2026)
- [Frontend-Assisted Large Audio-Language Models for SLT2026 SmartGlasses Challenge](https://mlsanity.com/paper/2610.13282/) (2026)

## Abstract (arXiv)

Self-supervised models such as WavLM have demonstrated strong performance for neural speaker diarization. However, these models are typically pre-trained on single-channel recordings, limiting their effectiveness in multi-channel scenarios. Existing diarization systems built on these models often rely on DOVER-Lap to combine outputs from individual channels. Although effective, this approach incurs substantial computational overhead and fails to fully exploit spatial information. In this work, building on DiariZen, a pipeline that combines WavLM-based local endto-end neural diarization with speaker embedding clustering, we introduce a lightweight approach to make pre-trained WavLM spatially aware by inserting channel communication modules into the early layers. Our method is agnostic to both the number of microphone channels and array topologies, ensuring broad applicability. We further propose to fuse multi-channel speaker embeddings by leveraging spatial attention weights. Evaluations on five public datasets show consistent improvements over single-channel baselines and demonstrate superior performance and efficiency compared with DOVER-Lap. Our source code is publicly available at https://github.com/BUTSpeechFIT/DiariZen.