# end-to-end training of a large vocabulary end-to-end speech recognition system

arXiv: https://arxiv.org/abs/1912.11040 · HTML: https://mlsanity.com/paper/1912.11040/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2019.

## Built on by (most cited first, 5 of 5)

- [Multi-stage Progressive Compression of Conformer Transducer for On-device Speech Recognition](https://mlsanity.com/paper/2210.00169/) (2022)
- [Streaming end-to-end speech recognition with jointly trained neural feature enhancement](https://mlsanity.com/paper/2105.01254/) (2021)
- [Two-Pass End-to-End ASR Model Compression](https://mlsanity.com/paper/2201.02741/) (2022)
- [Semi-supervised transfer learning for language expansion of end-to-end speech recognition models to low-resource languages](https://mlsanity.com/paper/2111.10047/) (2021)
- [Macro-block dropout for improved regularization in training end-to-end speech recognition models](https://mlsanity.com/paper/2212.14149/) (2022)

## Abstract (arXiv)

In this paper, we present an end-to-end training framework for building state-of-the-art end-to-end speech recognition systems. Our training system utilizes a cluster of Central Processing Units(CPUs) and Graphics Processing Units (GPUs). The entire data reading, large scale data augmentation, neural network parameter updates are all performed "on-the-fly". We use vocal tract length perturbation [1] and an acoustic simulator [2] for data augmentation. The processed features and labels are sent to the GPU cluster. The Horovod allreduce approach is employed to train neural network parameters. We evaluated the effectiveness of our system on the standard Librispeech corpus [3] and the 10,000-hr anonymized Bixby English dataset. Our end-to-end speech recognition system built using this training infrastructure showed a 2.44 % WER on test-clean of the LibriSpeech test set after applying shallow fusion with a Transformer language model (LM). For the proprietary English Bixby open domain test set, we obtained a WER of 7.92 % using a Bidirectional Full Attention (BFA) end-to-end model after applying shallow fusion with an RNN-LM. When the monotonic chunckwise attention (MoCha) based approach is employed for streaming speech recognition, we obtained a WER of 9.95 % on the same Bixby open domain test set.