# Speaker Diarization with Region Proposal Network

arXiv: https://arxiv.org/abs/2002.06220 · HTML: https://mlsanity.com/paper/2002.06220/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2020.

## Built on by (most cited first, 16 of 16)

- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [AVA-AVD: Audio-Visual Speaker Diarization in the Wild](https://mlsanity.com/paper/2111.14448/) (2021)
- [Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR](https://mlsanity.com/paper/2110.03151/) (2021)
- [End-to-End Diarization for Variable Number of Speakers with Local-Global Networks and Discriminative Speaker Embeddings](https://mlsanity.com/paper/2105.02096/) (2021)
- [Using Active Speaker Faces for Diarization in TV shows](https://mlsanity.com/paper/2203.15961/) (2022)
- [Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator](https://mlsanity.com/paper/2305.16263/) (2023)
- [Contrastive Separative Coding for Self-supervised Representation Learning](https://mlsanity.com/paper/2103.00816/) (2021)
- [The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines](https://mlsanity.com/paper/2208.08042/) (2022)
- [Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding with Sequence-to-Sequence Architecture](https://mlsanity.com/paper/2309.09180/) (2023)
- [An Investigation Into Explainable Audio Hate Speech Detection](https://mlsanity.com/paper/2408.06065/) (2024)
- [Tune-In: Training Under Negative Environments with Interference for Attention Networks Simulating Cocktail Party Effect](https://mlsanity.com/paper/2103.01461/) (2021)
- [End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations](https://mlsanity.com/paper/2303.12002/) (2023)
- [Online speaker diarization of meetings guided by speech separation](https://mlsanity.com/paper/2402.00067/) (2024)
- [Domain-Aware Speaker Diarization On African-Accented English](https://mlsanity.com/paper/2509.21554/) (2025)
- [Speaker Diarization using Two-pass Leave-One-Out Gaussian PLDA Clustering of DNN Embeddings](https://mlsanity.com/paper/2104.02469/) (2021)
- [Advancing the dimensionality reduction of speaker embeddings for speaker diarisation: disentangling noise and informing speech activity](https://mlsanity.com/paper/2110.03380/) (2021)

## Abstract (arXiv)

Speaker diarization is an important pre-processing step for many speech applications, and it aims to solve the "who spoke when" problem. Although the standard diarization systems can achieve satisfactory results in various scenarios, they are composed of several independently-optimized modules and cannot deal with the overlapped speech. In this paper, we propose a novel speaker diarization method: Region Proposal Network based Speaker Diarization (RPNSD). In this method, a neural network generates overlapped speech segment proposals, and compute their speaker embeddings at the same time. Compared with standard diarization systems, RPNSD has a shorter pipeline and can handle the overlapped speech. Experimental results on three diarization datasets reveal that RPNSD achieves remarkable improvements over the state-of-the-art x-vector baseline.