mlsanityData to 14 Oct 2026

Serialized Output Training by Learned Dominance

arXiv 2407.03966 ↗First posted 4 Jul 2024cs.SD cs.AI eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

Serialized Output Training (SOT) has showcased state-of-the-art performance in multi-talker speech recognition by sequentially decoding the speech of individual speakers. To address the challenging label-permutation issue, prior methods have relied on either the Permutation Invariant Training (PIT) or the time-based First-In-First-Out (FIFO) rule. This study presents a model-based serialization strategy that incorporates an auxiliary module into the Attention Encoder-Decoder architecture, autonomously identifying the crucial factors to order the output sequence of the speech components in multi-talker speech. Experiments conducted on the LibriSpeech and LibriMix databases reveal that our approach significantly outperforms the PIT and FIFO baselines in both 2-mix and 3-mix scenarios. Further analysis shows that the serialization module identifies dominant speech components in a mixture by factors including loudness and gender, and orders speech components based on the dominance score.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

21 of 30 references.

References · 21

  1. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2015Listen, Attend and Spellcited by 543
  4. 2015Attention-Based Models for Speech Recognitioncited by 401
  5. 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 211
  6. 2016Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separationcited by 191
  7. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 187
  8. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 155
  9. 2019Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of Viewcited by 113
  10. 2019Transformer-based Acoustic Modeling for Hybrid Speech Recognitioncited by 88
  11. 2020Serialized Output Training for End-to-End Overlapped Speech Recognitioncited by 87
  12. 2014End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Resultscited by 68
9 more
  1. 2020Joint Speaker Counting, Speech Recognition, and Speaker Identification for Overlapped Speech of Any Number of Speakerscited by 41
  2. 2021Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphonecited by 33
  3. 2020Streaming end-to-end multi-talker speech recognitioncited by 29
  4. 2017Single-Channel Multi-talker Speech Recognition with Permutation Invariant Trainingcited by 16
  5. 2021Conformer-based Hybrid ASR System for Switchboard Datasetcited by 15
  6. 2023A Sidecar Separator Can Convert a Single-Talker Speech Recognition System to a Multi-Talker Onecited by 13
  7. 2024Cross-Speaker Encoding Network for Multi-Talker Speech Recognitioncited by 9
  8. 2021Investigation of Practical Aspects of Single Channel Speech Separation for ASRcited by 8
  9. 2023End-to-End Joint Target and Non-Target Speakers ASRcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 2.

5 papers.

  1. 2024Large Language Model Can Transcribe Speech in Multi-Talker Scenarios with Versatile Instructionscited by 17
  2. 2024Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systemscited by 15
  3. 2024Disentangling Speakers in Multi-Talker Speech Recognition with Speaker-Aware CTCcited by 4
  4. 2025Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audiocited by 4
  5. 2025GLAD: Global-Local Aware Dynamic Mixture-of-Experts for Multi-Talker ASRcited by 1