# Speaker Adaptation for End-to-End CTC Models

arXiv: https://arxiv.org/abs/1901.01239 · HTML: https://mlsanity.com/paper/1901.01239/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jan 2019.

## Built on by (most cited first, 5 of 5)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [Multiple-hypothesis CTC-based semi-supervised adaptation of end-to-end speech recognition](https://mlsanity.com/paper/2103.15515/) (2021)
- [Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation](https://mlsanity.com/paper/2407.06310/) (2024)
- [Target Speaker Lipreading by Audio-Visual Self-Distillation Pretraining and Speaker Adaptation](https://mlsanity.com/paper/2502.05758/) (2025)
- [Speaker-Adapted End-to-End Visual Speech Recognition for Continuous Spanish](https://mlsanity.com/paper/2311.12480/) (2023)

## Abstract (arXiv)

We propose two approaches for speaker adaptation in end-to-end (E2E) automatic speech recognition systems. One is Kullback-Leibler divergence (KLD) regularization and the other is multi-task learning (MTL). Both approaches aim to address the data sparsity especially output target sparsity issue of speaker adaptation in E2E systems. The KLD regularization adapts a model by forcing the output distribution from the adapted model to be close to the unadapted one. The MTL utilizes a jointly trained auxiliary task to improve the performance of the main task. We investigated our approaches on E2E connectionist temporal classification (CTC) models with three different types of output units. Experiments on the Microsoft short message dictation task demonstrated that MTL outperforms KLD regularization. In particular, the MTL adaptation obtained 8.8% and 4.0% relative word error rate reductions (WERRs) for supervised and unsupervised adaptations for the word CTC model, and 9.6% and 3.8% relative WERRs for the mix-unit CTC model, respectively.