mlsanityData to 14 Oct 2026

BA-SOT: Boundary-Aware Serialized Output Training for Multi-Talker ASR

arXiv 2305.13716 ↗First posted 23 May 2023cs.SD cs.CL eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

The recently proposed serialized output training (SOT) simplifies multi-talker automatic speech recognition (ASR) by generating speaker transcriptions separated by a special token. However, frequent speaker changes can make speaker change prediction difficult. To address this, we propose boundary-aware serialized output training (BA-SOT), which explicitly incorporates boundary knowledge into the decoder via a speaker change detection task and boundary constraint loss. We also introduce a two-stage connectionist temporal classification (CTC) strategy that incorporates token-level SOT CTC to restore temporal context information. Besides typical character error rate (CER), we introduce utterance-dependent character error rate (UD-CER) to further measure the precision of speaker change prediction. Compared to original SOT, BA-SOT reduces CER/UD-CER by 5.1%/14.0%, and leveraging a pre-trained ASR model for BA-SOT model initialization further reduces CER/UD-CER by 8.4%/19.9%.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 28 of its 36 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

28 of 36 references.

References · 28

  1. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  2. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  3. 2016Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separationcited by 191
  4. 2020CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordingscited by 132
  5. 2018The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselinescited by 92
  6. 2020Serialized Output Training for End-to-End Overlapped Speech Recognitioncited by 87
  7. 2020Continuous speech separation: dataset and analysiscited by 74
  8. 2021M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challengecited by 71
  9. 2021AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenariocited by 60
  10. 2019MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognitioncited by 44
  11. 2022Streaming Multi-Talker ASR with Token-Level Serialized Output Trainingcited by 37
  12. 2020Continuous Speech Separation with Conformercited by 36
16 more
  1. 2017Recognizing Multi-talker Speech with Permutation Invariant Trainingcited by 33
  2. 2018A Purely End-to-end System for Multi-speaker Speech Recognitioncited by 30
  3. 2018End-to-End Monaural Multi-speaker ASR System without Pretrainingcited by 26
  4. 2022CTC Alignments Improve Autoregressive Translationcited by 25
  5. 2017Single-Channel Multi-talker Speech Recognition with Permutation Invariant Trainingcited by 16
  6. 2021Deep Learning based Multi-Source Localization with Source Splitting and its Effectiveness in Multi-Talker Speech Recognitioncited by 14
  7. 2021Multi-Speaker ASR Combining Non-Autoregressive Conformer CTC and Conditional Speaker Chaincited by 12
  8. 2021A Real-time Speaker Diarization System Based on Spatial Spectrumcited by 11
  9. 2022Summary On The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Grand Challengecited by 11
  10. 2022Simulating realistic speech overlaps improves multi-talker ASRcited by 10
  11. 2020Sequence to Multi-Sequence Learning via Conditional Chain Mapping for Mixture Signalscited by 5
  12. 2020An End-to-end Architecture of Online Multi-channel Speech Separationcited by 5
  13. 2021Graph-PIT: Generalized permutation invariant training for continuous separation of arbitrary numbers of speakerscited by 5
  14. 2020Speaker-Conditional Chain Model for Speech Separation and Extractioncited by 4
  15. 2021Location-based training for multi-channel talker-independent speaker separationcited by 4
  16. 2022MFCCA:Multi-Frame Cross-Channel attention for multi-speaker ASR in Multi-party meeting scenariocited by 4

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 2.

5 papers.

  1. 2024Cross-Speaker Encoding Network for Multi-Talker Speech Recognitioncited by 9
  2. 2024Disentangling Speakers in Multi-Talker Speech Recognition with Speaker-Aware CTCcited by 4
  3. 2025Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audiocited by 4
  4. 2025Speaker-Distinguishable CTC: Learning Speaker Distinction Using CTC for Multi-Talker Speech Recognitioncited by 3
  5. 2023End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translationcited by 2