Fully Supervised Speaker Diarization
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose a fully supervised speaker diarization approach, named unbounded interleaved-state recurrent neural networks (UIS-RNN). Given extracted speaker-discriminative embeddings (a.k.a. d-vectors) from input utterances, each individual speaker is modeled by a parameter-sharing RNN, while the RNN states for different speakers interleave in the time domain. This RNN is naturally integrated with a distance-dependent Chinese restaurant process (ddCRP) to accommodate an unknown number of speakers. Our system is fully supervised and is able to learn from examples where time-stamped speaker labels are annotated. We achieved a 7.6% diarization error rate on NIST SRE 2000 CALLHOME, which is better than the state-of-the-art method using spectral clustering. Moreover, our method decodes in an online fashion while most state-of-the-art systems rely on offline clustering.
Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Oct 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 13; 2022: 12; 2023: 6; 2024: 9; 2025: 3; 2026: 3.
46 papers.
Sort:
By area:
- 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 122
- 2021MAAS: Multi-modal Assignation for Active Speaker Detectioncited by 35
- 2021Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detectioncited by 31
- 2021AVA-AVD: Audio-Visual Speaker Diarization in the Wildcited by 24
- 2021Hear Me Out: Fusional Approaches for Audio Augmented Temporal Action Localizationcited by 21
- 2024DiarizationLM: Speaker Diarization Post-Processing with Large Language Modelscited by 16
- 2021Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractorscited by 14
- 2023Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correctioncited by 14
- 2021Streaming Multi-talker Speech Recognition with Joint Speaker Identificationcited by 12
- 2021A Real-time Speaker Diarization System Based on Spatial Spectrumcited by 11
- 2022Highly Efficient Real-Time Streaming and Fully On-Device Speaker Diarization with Multi-Stage Clusteringcited by 10
- 2024The VoxCeleb Speaker Recognition Challenge: A Retrospectivecited by 9
34 more
- 2024Look, Listen and Recognise: Character-Aware Audio-Visual Subtitlingcited by 8
- 2024LLM-based speaker diarization correction: A generalizable approachcited by 8
- 2021End-to-End Diarization for Variable Number of Speakers with Local-Global Networks and Discriminative Speaker Embeddingscited by 7
- 2022Improving the Naturalness of Simulated Conversations for End-to-End Neural Diarizationcited by 7
- 2022Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractorscited by 7
- 2021DIVE: End-to-end Speech Diarization via Iterative Speaker Embeddingcited by 5
- 2022In search of strong embedding extractors for speaker diarisationcited by 5
- 2022ASR-Aware End-to-end Neural Diarizationcited by 4
- 2023Multi-stage Learning for Radar Pulse Activity Segmentationcited by 4
- 2021APES: Audiovisual Person Search in Untrimmed Videocited by 3
- 2022Self-supervised Speaker Diarizationcited by 3
- 2022The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselinescited by 3
- 2025Character-Centric Understanding of Animated Moviescited by 3
- 2021Speaker attribution with voice profiles by graph-based semi-supervised learningcited by 2
- 2021Content-Aware Speaker Embeddings for Speaker Diarisationcited by 2
- 2022Tight integration of neural- and clustering-based diarization through deep unfolding of infinite Gaussian mixture modelcited by 2
- 2024Online speaker diarization of meetings guided by speech separationcited by 2
- 2024Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarizationcited by 2
- 2024Character-aware audio-visual subtitling in contextcited by 2
- 2025Domain-Aware Speaker Diarization On African-Accented Englishcited by 2
- 2023Speaker Diarization of Scripted Audiovisual Contentcited by 1
- 2023Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Networkcited by 1
- 2023Detecting agreement in multi-party dialogue: evaluating speaker diarisation versus a procedural baseline to enhance user engagementcited by 1
- 2024Systematic Evaluation of Online Speaker Diarization Systems Regarding their Latencycited by 1
- 2026MK-SGC-SC: Multiple Kernel Guided Sparse Graph Construction in Spectral Clustering for Unsupervised Speaker Diarizationcited by 1
- 2022Generation of Speaker Representations Using Heterogeneous Training Batch Assemblycited by 0
- 2022Unsupervised Speaker Diarization that is Agnostic to Language, Overlap-Aware, and Tuning Freecited by 0
- 2022Chronological Self-Training for Real-Time Speaker Diarizationcited by 0
- 2022Mutual Learning of Single- and Multi-Channel End-to-End Neural Diarizationcited by 0
- 2023Implicit Self-supervised Language Representation for Spoken Language Diarizationcited by 0
- 2024A Review of Common Online Speaker Diarization Methodscited by 0
- 2025O-EENC-SD: Efficient Online End-to-End Neural Clustering for Speaker Diarizationcited by 0
- 2026CineSRD: Leveraging Visual, Acoustic, and Linguistic Cues for Open-World Visual Media Speaker Diarizationcited by 0
- 2026A Novel Automatic Framework for Speaker Drift Detection in Synthesized Speechcited by 0
- 2026A Novel Automatic Framework for Speaker Drift Detection in Synthesized Speech7 Apr 2026
- 2026CineSRD: Leveraging Visual, Acoustic, and Linguistic Cues for Open-World Visual Media Speaker Diarization17 Mar 2026
- 2026MK-SGC-SC: Multiple Kernel Guided Sparse Graph Construction in Spectral Clustering for Unsupervised Speaker Diarization24 Jan 2026
- 2025O-EENC-SD: Efficient Online End-to-End Neural Clustering for Speaker Diarization17 Dec 2025
- 2025Domain-Aware Speaker Diarization On African-Accented English25 Sep 2025
- 2025Character-Centric Understanding of Animated Movies15 Sep 2025
- 2024Character-aware audio-visual subtitling in context14 Oct 2024
- 2024The VoxCeleb Speaker Recognition Challenge: A Retrospective27 Aug 2024
- 2024Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarization22 Aug 2024
- 2024Systematic Evaluation of Online Speaker Diarization Systems Regarding their Latency5 Jul 2024
- 2024A Review of Common Online Speaker Diarization Methods20 Jun 2024
- 2024LLM-based speaker diarization correction: A generalizable approach7 Jun 2024
34 more
- 2024Online speaker diarization of meetings guided by speech separation30 Jan 2024
- 2024Look, Listen and Recognise: Character-Aware Audio-Visual Subtitling22 Jan 2024
- 2024DiarizationLM: Speaker Diarization Post-Processing with Large Language Models7 Jan 2024
- 2023Multi-stage Learning for Radar Pulse Activity Segmentation15 Dec 2023
- 2023Detecting agreement in multi-party dialogue: evaluating speaker diarisation versus a procedural baseline to enhance user engagement6 Nov 2023
- 2023Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Network15 Sep 2023
- 2023Implicit Self-supervised Language Representation for Spoken Language Diarization21 Aug 2023
- 2023Speaker Diarization of Scripted Audiovisual Content4 Aug 2023
- 2023Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correction15 Jun 2023
- 2022In search of strong embedding extractors for speaker diarisation26 Oct 2022
- 2022Highly Efficient Real-Time Streaming and Fully On-Device Speaker Diarization with Multi-Stage Clustering25 Oct 2022
- 2022Mutual Learning of Single- and Multi-Channel End-to-End Neural Diarization7 Oct 2022
- 2022The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines17 Aug 2022
- 2022Chronological Self-Training for Real-Time Speaker Diarization5 Aug 2022
- 2022Unsupervised Speaker Diarization that is Agnostic to Language, Overlap-Aware, and Tuning Free25 Jul 2022
- 2022Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors6 Jun 2022
- 2022Improving the Naturalness of Simulated Conversations for End-to-End Neural Diarization24 Apr 2022
- 2022Self-supervised Speaker Diarization8 Apr 2022
- 2022Generation of Speaker Representations Using Heterogeneous Training Batch Assembly30 Mar 2022
- 2022Tight integration of neural- and clustering-based diarization through deep unfolding of infinite Gaussian mixture model14 Feb 2022
- 2022ASR-Aware End-to-end Neural Diarization2 Feb 2022
- 2021AVA-AVD: Audio-Visual Speaker Diarization in the Wild29 Nov 2021
- 2021Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection23 Sep 2021
- 2021A Real-time Speaker Diarization System Based on Spatial Spectrum20 Jul 2021
- 2021Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors4 Jul 2021
- 2021Hear Me Out: Fusional Approaches for Audio Augmented Temporal Action Localization27 Jun 2021
- 2021APES: Audiovisual Person Search in Untrimmed Video3 Jun 2021
- 2021DIVE: End-to-end Speech Diarization via Iterative Speaker Embedding28 May 2021
- 2021End-to-End Diarization for Variable Number of Speakers with Local-Global Networks and Discriminative Speaker Embeddings5 May 2021
- 2021Streaming Multi-talker Speech Recognition with Joint Speaker Identification5 Apr 2021
- 2021Content-Aware Speaker Embeddings for Speaker Diarisation12 Feb 2021
- 2021Speaker attribution with voice profiles by graph-based semi-supervised learning6 Feb 2021
- 2021A Review of Speaker Diarization: Recent Advances with Deep Learning24 Jan 2021
- 2021MAAS: Multi-modal Assignation for Active Speaker Detection11 Jan 2021
- 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 122
- 2021MAAS: Multi-modal Assignation for Active Speaker Detectioncited by 35
- 2021Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detectioncited by 31
- 2021AVA-AVD: Audio-Visual Speaker Diarization in the Wildcited by 24
- 2021Hear Me Out: Fusional Approaches for Audio Augmented Temporal Action Localizationcited by 21
- 2024DiarizationLM: Speaker Diarization Post-Processing with Large Language Modelscited by 16
- 2021Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractorscited by 14
- 2023Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correctioncited by 14
- 2021Streaming Multi-talker Speech Recognition with Joint Speaker Identificationcited by 12
- 2021A Real-time Speaker Diarization System Based on Spatial Spectrumcited by 11
- 2022Highly Efficient Real-Time Streaming and Fully On-Device Speaker Diarization with Multi-Stage Clusteringcited by 10
- 2024The VoxCeleb Speaker Recognition Challenge: A Retrospectivecited by 9
12 most cited of 45 All 45 (JSON)
- 2026CineSRD: Leveraging Visual, Acoustic, and Linguistic Cues for Open-World Visual Media Speaker Diarization17 Mar 2026
- 2026MK-SGC-SC: Multiple Kernel Guided Sparse Graph Construction in Spectral Clustering for Unsupervised Speaker Diarization24 Jan 2026
- 2025O-EENC-SD: Efficient Online End-to-End Neural Clustering for Speaker Diarization17 Dec 2025
- 2025Domain-Aware Speaker Diarization On African-Accented English25 Sep 2025
- 2025Character-Centric Understanding of Animated Movies15 Sep 2025
- 2024Character-aware audio-visual subtitling in context14 Oct 2024
- 2024The VoxCeleb Speaker Recognition Challenge: A Retrospective27 Aug 2024
- 2024Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarization22 Aug 2024
- 2024Systematic Evaluation of Online Speaker Diarization Systems Regarding their Latency5 Jul 2024
- 2024A Review of Common Online Speaker Diarization Methods20 Jun 2024
- 2024LLM-based speaker diarization correction: A generalizable approach7 Jun 2024
- 2024Online speaker diarization of meetings guided by speech separation30 Jan 2024
12 newest of 45 All 45 (JSON)