# A Real-time Speaker Diarization System Based on Spatial Spectrum

arXiv: https://arxiv.org/abs/2107.09321 · HTML: https://mlsanity.com/paper/2107.09321/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 20 references that are in the map.
- First posted on arXiv 20 Jul 2021.

## Builds on (most cited first, 5 of 5)

- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [Speaker Diarization with LSTM](https://mlsanity.com/paper/1710.10468/) (2017)
- [Fully Supervised Speaker Diarization](https://mlsanity.com/paper/1810.04719/) (2018)
- [BUT System for the Second DIHARD Speech Diarization Challenge](https://mlsanity.com/paper/2002.11356/) (2020)
- [Cracking the cocktail party problem by multi-beam deep attractor network](https://mlsanity.com/paper/1803.10924/) (2018)

## Built on by (most cited first, 11 of 11)

- [3D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglement](https://mlsanity.com/paper/2306.15354/) (2023)
- [A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings](https://mlsanity.com/paper/2203.16834/) (2022)
- [Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization](https://mlsanity.com/paper/2305.12927/) (2023)
- [BA-SOT: Boundary-Aware Serialized Output Training for Multi-Talker ASR](https://mlsanity.com/paper/2305.13716/) (2023)
- [The Volcspeech system for the ICASSP 2022 multi-channel multi-party meeting transcription challenge](https://mlsanity.com/paper/2202.04261/) (2022)
- [Reformulating Speaker Diarization as Community Detection With Emphasis On Topological Structure](https://mlsanity.com/paper/2204.12112/) (2022)
- [BeamTransformer: Microphone Array-based Overlapping Speech Detection](https://mlsanity.com/paper/2109.04049/) (2021)
- [Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarization](https://mlsanity.com/paper/2408.12102/) (2024)
- [Speaker Embedding-aware Neural Diarization for Flexible Number of Speakers with Textual Information](https://mlsanity.com/paper/2111.13694/) (2021)
- [Iterative Sound Source Localization for Unknown Number of Sources](https://mlsanity.com/paper/2206.12273/) (2022)
- [CineSRD: Leveraging Visual, Acoustic, and Linguistic Cues for Open-World Visual Media Speaker Diarization](https://mlsanity.com/paper/2603.16966/) (2026)

## Abstract (arXiv)

In this paper we describe a speaker diarization system that enables localization and identification of all speakers present in a conversation or meeting. We propose a novel systematic approach to tackle several long-standing challenges in speaker diarization tasks: (1) to segment and separate overlapping speech from two speakers; (2) to estimate the number of speakers when participants may enter or leave the conversation at any time; (3) to provide accurate speaker identification on short text-independent utterances; (4) to track down speakers movement during the conversation; (5) to detect speaker change incidence real-time. First, a differential directional microphone array-based approach is exploited to capture the target speakers' voice in far-field adverse environment. Second, an online speaker-location joint clustering approach is proposed to keep track of speaker location. Third, an instant speaker number detector is developed to trigger the mechanism that separates overlapped speech. The results suggest that our system effectively incorporates spatial information and achieves significant gains.