# Bypass Temporal Classification: Weakly Supervised Automatic Speech Recognition with Imperfect Transcripts

arXiv: https://arxiv.org/abs/2306.01031 · HTML: https://mlsanity.com/paper/2306.01031/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding](https://mlsanity.com/paper/1507.08240/) (2015)
- [Differentiable Weighted Finite-State Transducers](https://mlsanity.com/paper/2010.01003/) (2020)
- [CTC Variations Through New WFST Topologies](https://mlsanity.com/paper/2110.03098/) (2021)
- [Star Temporal Classification: Sequence Classification with Partially Labeled Data](https://mlsanity.com/paper/2201.12208/) (2022)
- [Handling Noisy Labels for Robustly Learning from Self-Training Data for Low-Resource Sequence Labeling](https://mlsanity.com/paper/1903.12008/) (2019)

## Built on by (most cited first, 10 of 10)

- [Updated Corpora and Benchmarks for Long-Form Speech Recognition](https://mlsanity.com/paper/2309.15013/) (2023)
- [Learning from Flawed Data: Weakly Supervised Automatic Speech Recognition](https://mlsanity.com/paper/2309.15796/) (2023)
- [Token-Weighted RNN-T for Learning from Flawed Data](https://mlsanity.com/paper/2406.18108/) (2024)
- [Fast Streaming Transducer ASR Prototyping via Knowledge Distillation with Whisper](https://mlsanity.com/paper/2409.13499/) (2024)
- [RNN-Transducer-based Losses for Speech Recognition on Noisy Targets](https://mlsanity.com/paper/2504.06963/) (2025)
- [WST: Weakly Supervised Transducer for Automatic Speech Recognition](https://mlsanity.com/paper/2511.04035/) (2025)
- [Graph Connectionist Temporal Classification for Phoneme Recognition](https://mlsanity.com/paper/2509.05399/) (2025)
- [Modeling Overlapped Speech with Shuffles](https://mlsanity.com/paper/2603.17769/) (2026)
- [Constrained CTC Decoding for Efficient Diacritic Restoration](https://mlsanity.com/paper/2607.18946/) (2026)
- [A Training Criterion with Token-Level Tolerance to Transcription Ambiguity for Automatic Speech Recognition](https://mlsanity.com/paper/2609.30160/) (2026)

## Abstract (arXiv)

This paper presents a novel algorithm for building an automatic speech recognition (ASR) model with imperfect training data. Imperfectly transcribed speech is a prevalent issue in human-annotated speech corpora, which degrades the performance of ASR models. To address this problem, we propose Bypass Temporal Classification (BTC) as an expansion of the Connectionist Temporal Classification (CTC) criterion. BTC explicitly encodes the uncertainties associated with transcripts during training. This is accomplished by enhancing the flexibility of the training graph, which is implemented as a weighted finite-state transducer (WFST) composition. The proposed algorithm improves the robustness and accuracy of ASR systems, particularly when working with imprecisely transcribed speech corpora. Our implementation will be open-sourced.