# Multiple-hypothesis CTC-based semi-supervised adaptation of end-to-end speech recognition

arXiv: https://arxiv.org/abs/2103.15515 · HTML: https://mlsanity.com/paper/2103.15515/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 28 references that are in the map.
- First posted on arXiv 29 Mar 2021.

## Builds on (most cited first, 8 of 8)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [Speaker Adaptation for Attention-Based End-to-End Speech Recognition](https://mlsanity.com/paper/1911.03762/) (2019)
- [Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASR](https://mlsanity.com/paper/2002.06165/) (2020)
- [Speaker Adaptation for End-to-End CTC Models](https://mlsanity.com/paper/1901.01239/) (2019)
- [End-to-end Adaptation with Backpropagation through WFST for On-device Speech Recognition System](https://mlsanity.com/paper/1905.07149/) (2019)

## Built on by (most cited first, 4 of 4)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [Alternative Pseudo-Labeling for Semi-Supervised Automatic Speech Recognition](https://mlsanity.com/paper/2308.06547/) (2023)
- [Multiple-hypothesis RNN-T Loss for Unsupervised Fine-tuning and Self-training of Neural Transducer](https://mlsanity.com/paper/2207.14736/) (2022)
- [Graph Connectionist Temporal Classification for Phoneme Recognition](https://mlsanity.com/paper/2509.05399/) (2025)

## Abstract (arXiv)

This paper proposes an adaptation method for end-to-end speech recognition. In this method, multiple automatic speech recognition (ASR) 1-best hypotheses are integrated in the computation of the connectionist temporal classification (CTC) loss function. The integration of multiple ASR hypotheses helps alleviating the impact of errors in the ASR hypotheses to the computation of the CTC loss when ASR hypotheses are used. When being applied in semi-supervised adaptation scenarios where part of the adaptation data do not have labels, the CTC loss of the proposed method is computed from different ASR 1-best hypotheses obtained by decoding the unlabeled adaptation data. Experiments are performed in clean and multi-condition training scenarios where the CTC-based end-to-end ASR systems are trained on Wall Street Journal (WSJ) clean training data and CHiME-4 multi-condition training data, respectively, and tested on Aurora-4 test data. The proposed adaptation method yields 6.6% and 5.8% relative word error rate (WER) reductions in clean and multi-condition training scenarios, respectively, compared to a baseline system which is adapted with part of the adaptation data having manual transcriptions using back-propagation fine-tuning.