mlsanityData to 7 Oct 2026

Joint Speaker Counting, Speech Recognition, and Speaker Identification for Overlapped Speech of Any Number of Speakers

arXiv 2006.10930 ↗First posted 19 Jun 2020eess.AS cs.CL cs.SDcited by 41

Not on the map. Why?

Abstract (arXiv)

We propose an end-to-end speaker-attributed automatic speech recognition model that unifies speaker counting, speech recognition, and speaker identification on monaural overlapped speech. Our model is built on serialized output training (SOT) with attention-based encoder-decoder, a recently proposed method for recognizing overlapped speech comprising an arbitrary number of speakers. We extend SOT by introducing a speaker inventory as an auxiliary input to produce speaker labels as well as multi-speaker transcriptions. All model parameters are optimized by speaker-attributed maximum mutual information criterion, which represents a joint probability for overlapped speech recognition and speaker identification. Experiments on LibriSpeech corpus show that our proposed method achieves significantly better speaker-attributed word error rate than the baseline that separately performs overlapped speech recognition and speaker identification.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 7; 2023: 5; 2024: 9; 2025: 5; 2026: 2.

41 papers.

Sort:

By area:

  1. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  2. 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 122
  3. 2022Streaming Multi-Talker ASR with Token-Level Serialized Output Trainingcited by 37
  4. 2021End-to-End Speaker-Attributed ASR with Transformercited by 35
  5. 2021Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphonecited by 33
  6. 2021Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASRcited by 22
  7. 2024DiarizationLM: Speaker Diarization Post-Processing with Large Language Modelscited by 16
  8. 2022Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddingscited by 14
  9. 2021Streaming Multi-talker Speech Recognition with Joint Speaker Identificationcited by 12
  10. 2023One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognitioncited by 12
  11. 2024Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systemscited by 12
  12. 2025CoVoMix2: Advancing Zero-Shot Dialogue Generation with Fully Non-Autoregressive Flow Matchingcited by 11
29 more
  1. 2021A Comparative Study of Modular and Joint Approaches for Speaker-Attributed ASR on Monaural Long-Form Audiocited by 10
  2. 2021Multi-turn RNN-T for streaming recognition of multi-party speechcited by 10
  3. 2023Conformer-based Target-Speaker Automatic Speech Recognition for Single-Channel Audiocited by 10
  4. 2024Cross-Speaker Encoding Network for Multi-Talker Speech Recognitioncited by 9
  5. 2021Consistent Training and Decoding For End-to-end Speech Recognition Using Lattice-free MMIcited by 8
  6. 2021Hypothesis Stitcher for End-to-End Speaker-attributed ASR on Long-form Multi-talker Recordingscited by 7
  7. 2023Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separatorcited by 7
  8. 2026D-ORCA: Dialogue-Centric Optimization for Robust Audio-Visual Captioningcited by 6
  9. 2021Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speechcited by 5
  10. 2022A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetingscited by 5
  11. 2024Serialized Output Training by Learned Dominancecited by 5
  12. 2021Multi-user VoiceFilter-Lite via Attentive Speaker Embeddingcited by 4
  13. 2021Unified Autoregressive Modeling for Joint End-to-End Multi-Talker Overlapped Speech Recognition and Speaker Attribute Estimationcited by 4
  14. 2024Disentangling Speakers in Multi-Talker Speech Recognition with Speaker-Aware CTCcited by 4
  15. 2025Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audiocited by 4
  16. 2022The RoyalFlush System of Speech Recognition for M2MeT Challengecited by 3
  17. 2022Separator-Transducer-Segmenter: Streaming Recognition and Segmentation of Multi-party Speechcited by 3
  18. 2023End-to-end Multichannel Speaker-Attributed ASR: Speaker Guided Decoder and Input Feature Analysiscited by 3
  19. 2024Improving Speaker Assignment in Speaker-Attributed ASR for Real Meeting Applicationscited by 3
  20. 2025The Impact of Automatic Speech Transcription on Speaker Attributioncited by 2
  21. 2022Closing the Gap between Single-User and Multi-User VoiceFilter-Litecited by 1
  22. 2023Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Networkcited by 1
  23. 2024Unsupervised Speaker Diarization in Distributed IoT Networks Using Federated Learningcited by 1
  24. 2024Hypothesis Clustering and Merging: Novel MultiTalker Speech Recognition with Speaker Tokenscited by 1
  25. 2025GLAD: Global-Local Aware Dynamic Mixture-of-Experts for Multi-Talker ASRcited by 1
  26. 2022Coarse-to-Fine Recursive Speech Separation for Unknown Number of Speakerscited by 0
  27. 2024Joint Beamforming and Speaker-Attributed ASR for Real Distant-Microphone Meeting Transcriptioncited by 0
  28. 2025Improving Practical Aspects of End-to-End Multi-Talker Speech Recognition for Online and Offline Scenarioscited by 0
  29. 2026Who Said What, and Will It Be Remembered? Evaluating Persistent Speaker Attribution Across Meetingscited by 0