mlsanityData to 14 Oct 2026

Learning from Flawed Data: Weakly Supervised Automatic Speech Recognition

arXiv 2309.15796 ↗First posted 26 Sep 2023eess.AS cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Training automatic speech recognition (ASR) systems requires large amounts of well-curated paired data. However, human annotators usually perform "non-verbatim" transcription, which can result in poorly trained models. In this paper, we propose Omni-temporal Classification (OTC), a novel training criterion that explicitly incorporates label uncertainties originating from such weak supervision. This allows the model to effectively learn speech-text alignments while accommodating errors present in the training transcripts. OTC extends the conventional CTC objective for imperfect transcripts by leveraging weighted finite state transducers. Through experiments conducted on the LibriSpeech and LibriVox datasets, we demonstrate that training ASR models with OTC avoids performance degradation even with transcripts containing up to 70% errors, a scenario where CTC models fail completely. Our implementation is available at https://github.com/k2-fsa/icefall.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 26 Sep 2023.

Builds on

23 of 41 references.

References · 23

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  6. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  7. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 226
  8. 2021Unsupervised Speech Recognitioncited by 148
  9. 2019Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debatescited by 114
  10. 2015EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decodingcited by 73
  11. 2020On the Comparison of Popular End-to-End Models for Large Scale Speech Recognitioncited by 63
  12. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 45
11 more
  1. 2020Exploring Transformers for Large-Scale Speech Recognitioncited by 24
  2. 2018Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matchingcited by 20
  3. 2021CTC Variations Through New WFST Topologiescited by 17
  4. 2020Transformer in action: a comparative study of transformer-based acoustic models for large scale speech recognition applicationscited by 11
  5. 2021Deciphering Speech: a Zero-Resource Approach to Cross-Lingual Transfer in ASRcited by 11
  6. 2022Star Temporal Classification: Sequence Classification with Partially Labeled Datacited by 10
  7. 2023Bypass Temporal Classification: Weakly Supervised Automatic Speech Recognition with Imperfect Transcriptscited by 10
  8. 2023Alternative Pseudo-Labeling for Semi-Supervised Automatic Speech Recognitioncited by 6
  9. 2022EURO: ESPnet Unsupervised ASR Open-source Toolkitcited by 5
  10. 2019Lattice-based lightly-supervised acoustic model trainingcited by 3
  11. 2023HK-LegiCoST: Leveraging Non-Verbatim Transcripts for Speech Translationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 3.

8 papers.

  1. 2024Token-Weighted RNN-T for Learning from Flawed Datacited by 2
  2. 2025Advancing Arabic Speech Recognition Through Large-Scale Weakly Supervised Learningcited by 2
  3. 2025RNN-Transducer-based Losses for Speech Recognition on Noisy Targetscited by 1
  4. 2025WST: Weakly Supervised Transducer for Automatic Speech Recognitioncited by 1
  5. 2025Graph Connectionist Temporal Classification for Phoneme Recognitioncited by 0
  6. 2026Modeling Overlapped Speech with Shufflescited by 0
  7. 2026Constrained CTC Decoding for Efficient Diacritic Restorationcited by 0
  8. 2026A Training Criterion with Token-Level Tolerance to Transcription Ambiguity for Automatic Speech Recognitioncited by 0