mlsanityData to 7 Oct 2026

Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection

arXiv 2109.11641 ↗First posted 23 Sep 2021eess.AS cs.LG cs.SDcited by 31

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a novel speaker diarization system for streaming on-device applications. In this system, we use a transformer transducer to detect the speaker turns, represent each speaker turn by a speaker embedding, then cluster these embeddings with constraints from the detected speaker turns. Compared with conventional clustering-based diarization systems, our system largely reduces the computational cost of clustering due to the sparsity of speaker turns. Unlike other supervised speaker diarization systems which require annotations of time-stamped speaker labels for training, our system only requires including speaker turn tokens during the transcribing process, which largely reduces the human efforts involved in data collection.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 20 of its 41 references that are in the map. First posted on arXiv 23 Sep 2021.

Builds on

20 of 41 references.

References · 20

  1. 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
  2. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
  3. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  4. 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 122
  5. 2019End-to-End Neural Speaker Diarization with Permutation-Free Objectivescited by 86
  6. 2017Speaker Diarization with LSTMcited by 73
  7. 2019Transformer-Transducer: End-to-End Speech Recognition with Self-Attentioncited by 68
  8. 2017Deep Speaker: an End-to-End Neural Speaker Embedding Systemcited by 56
  9. 2019Joint Speech Recognition and Speaker Diarization via Sequence Transductioncited by 49
  10. 2019CN-CELEB: a challenging Chinese speaker recognition datasetcited by 47
  11. 2018Fully Supervised Speaker Diarizationcited by 46
  12. 2020Auto-Tuning Spectral Clustering for Speaker Diarization Using Normalized Maximum Eigengapcited by 35
8 more
  1. 2020Transformer Transducer: One Model Unifying Streaming and Non-streaming Speech Recognitioncited by 23
  2. 2019Discriminative Neural Clustering for Speaker Diarisationcited by 14
  3. 2018Multimodal Speaker Segmentation and Diarization using Lexical and Acoustic Cues via Sequence to Sequence Neural Networkscited by 12
  4. 2021Dr-Vectors: Decision Residual Networks and an Improved Loss for Speaker Recognitioncited by 10
  5. 2021Personalized Keyphrase Detection using Speaker and Environment Informationcited by 8
  6. 2021End-to-End Diarization for Variable Number of Speakers with Local-Global Networks and Discriminative Speaker Embeddingscited by 7
  7. 2021Multi-user VoiceFilter-Lite via Attentive Speaker Embeddingcited by 4
  8. 2018Links: A High-Dimensional Online Clustering Methodcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 8; 2024: 10; 2025: 2; 2026: 3.

31 papers.

Sort:

By area:

  1. 2021Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASRcited by 22
  2. 2024DiarizationLM: Speaker Diarization Post-Processing with Large Language Modelscited by 16
  3. 2023Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correctioncited by 14
  4. 2021BERTraffic: BERT-based Joint Speaker Role and Speaker Change Detection for Air Traffic Control Communicationscited by 13
  5. 2022Highly Efficient Real-Time Streaming and Fully On-Device Speaker Diarization with Multi-Stage Clusteringcited by 10
  6. 2024Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognitioncited by 10
  7. 2022Augmenting Transformer-Transducer Based Speaker Change Detection With Token-Level Training Losscited by 8
  8. 2024LLM-based speaker diarization correction: A generalizable approachcited by 8
  9. 2022Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractorscited by 7
  10. 2023DiariST: Streaming Speech Translation with Speaker Diarizationcited by 5
  11. 2023USM-SCD: Multilingual Speaker Change Detection Based on Large Pretrained Foundation Modelscited by 4
  12. 2022Separator-Transducer-Segmenter: Streaming Recognition and Segmentation of Multi-party Speechcited by 3
19 more
  1. 2023Improving Speaker Diarization using Semantic Information: Joint Pairwise Constraints Propagationcited by 3
  2. 2024AG-LSEC: Audio Grounded Lexical Speaker Error Correctioncited by 3
  3. 2022Exploring Sequence-to-Sequence Transformer-Transducer Models for Keyword Spottingcited by 2
  4. 2023Speaker and Language Change Detection using Wav2vec2 and Whispercited by 2
  5. 2023Improved Long-Form Speech Recognition by Jointly Modeling the Primary and Non-primary Speakerscited by 2
  6. 2024TokenVerse: Towards Unifying Speech and NLP Tasks via Transducer-based ASRcited by 2
  7. 2024Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarizationcited by 2
  8. 2022High-resolution embedding extractor for speaker diarisationcited by 1
  9. 2023Speaker Diarization of Scripted Audiovisual Contentcited by 1
  10. 2023Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Networkcited by 1
  11. 2024Joint vs Sequential Speaker-Role Detection and Automatic Speech Recognition for Air-traffic Controlcited by 1
  12. 2024Systematic Evaluation of Online Speaker Diarization Systems Regarding their Latencycited by 1
  13. 2025SEAL: Speaker Error Correction using Acoustic-conditioned Large Language Modelscited by 1
  14. 2024A Review of Common Online Speaker Diarization Methodscited by 0
  15. 2024Leveraging Speaker Embeddings in End-to-End Neural Diarization for Two-Speaker Scenarioscited by 0
  16. 2025TokenVerse++: Towards Flexible Multitask Learning with Dynamic Task Activationcited by 0
  17. 2026A Novel Automatic Framework for Speaker Drift Detection in Synthesized Speechcited by 0
  18. 2026Mitigating Speaker Leakage in Cascaded Multi-talker ASR with Diarization-based Transcript Correctioncited by 0
  19. 2026Training-Free Affinity Fusion of Neural and Embedding-Based Speaker Diarizationcited by 0