# Overlap-aware diarization: resegmentation using neural end-to-end overlapped speech detection

arXiv: https://arxiv.org/abs/1910.11646 · HTML: https://mlsanity.com/paper/1910.11646/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2019.

## Built on by (most cited first, 27 of 27)

- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR](https://mlsanity.com/paper/2110.03151/) (2021)
- [Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2107.01545/) (2021)
- [End-to-end Neural Diarization: From Transformer to Conformer](https://mlsanity.com/paper/2106.07167/) (2021)
- [Three-class Overlapped Speech Detection using a Convolutional Recurrent Neural Network](https://mlsanity.com/paper/2104.02878/) (2021)
- [Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2206.02432/) (2022)
- [Target Speaker Voice Activity Detection with Transformers and Its Integration with End-to-End Neural Diarization](https://mlsanity.com/paper/2208.13085/) (2022)
- [Overlapped speech and gender detection with WavLM pre-trained features](https://mlsanity.com/paper/2209.04167/) (2022)
- [In search of strong embedding extractors for speaker diarisation](https://mlsanity.com/paper/2210.14682/) (2022)
- [BeamTransformer: Microphone Array-based Overlapping Speech Detection](https://mlsanity.com/paper/2109.04049/) (2021)
- [End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations](https://mlsanity.com/paper/2303.12002/) (2023)
- [Joint speech and overlap detection: a benchmark over multiple audio setup and speech domains](https://mlsanity.com/paper/2307.13012/) (2023)
- [Compositional Clustering: Applications to Multi-Label Object Recognition and Speaker Identification](https://mlsanity.com/paper/2109.04160/) (2021)
- [An Explainable Proxy Model for Multiabel Audio Segmentation](https://mlsanity.com/paper/2401.08268/) (2024)
- [End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization](https://mlsanity.com/paper/2401.12850/) (2024)
- [A Semi-Automatic Approach to Create Large Gender- and Age-Balanced Speaker Corpora: Usefulness of Speaker Diarization & Identification](https://mlsanity.com/paper/2404.17552/) (2024)
- [ASoBO: Attentive Beamformer Selection for Distant Speaker Diarization in Meetings](https://mlsanity.com/paper/2406.03251/) (2024)
- [Towards Robust Overlapping Speech Detection: A Speaker-Aware Progressive Approach Using WavLM](https://mlsanity.com/paper/2505.23207/) (2025)
- [Unsupervised Speaker Diarization that is Agnostic to Language, Overlap-Aware, and Tuning Free](https://mlsanity.com/paper/2207.12504/) (2022)
- [Joint Speech Activity and Overlap Detection with Multi-Exit Architecture](https://mlsanity.com/paper/2209.11906/) (2022)
- [The Newsbridge -Telecom SudParis VoxCeleb Speaker Recognition Challenge 2022 System Description](https://mlsanity.com/paper/2301.07491/) (2023)
- [Towards Measuring and Scoring Speaker Diarization Fairness](https://mlsanity.com/paper/2302.09991/) (2023)
- [Multi-microphone Automatic Speech Segmentation in Meetings Based on Circular Harmonics Features](https://mlsanity.com/paper/2306.04268/) (2023)
- [Geodesic interpolation of frame-wise speaker embeddings for the diarization of meeting scenarios](https://mlsanity.com/paper/2401.03963/) (2024)
- [Conversational Rubert for Detecting Competitive Interruptions in ASR-Transcribed Dialogues](https://mlsanity.com/paper/2407.14940/) (2024)
- [Robust Target Speaker Diarization and Separation via Augmented Speaker Embedding Sampling](https://mlsanity.com/paper/2508.06393/) (2025)

## Abstract (arXiv)

We address the problem of effectively handling overlapping speech in a diarization system. First, we detail a neural Long Short-Term Memory-based architecture for overlap detection. Secondly, detected overlap regions are exploited in conjunction with a frame-level speaker posterior matrix to make two-speaker assignments for overlapped frames in the resegmentation step. The overlap detection module achieves state-of-the-art performance on the AMI, DIHARD, and ETAPE corpora. We apply overlap-aware resegmentation on AMI, resulting in a 20% relative DER reduction over the baseline system. While this approach is by no means an end-all solution to overlap-aware diarization, it reveals promising directions for handling overlap.