# Faster, Simpler and More Accurate Hybrid ASR Systems Using Wordpieces

arXiv: https://arxiv.org/abs/2005.09150 · HTML: https://mlsanity.com/paper/2005.09150/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2020.

## Built on by (most cited first, 9 of 9)

- [Unsupervised Speech Recognition](https://mlsanity.com/paper/2105.11084/) (2021)
- [Fast Text-Only Domain Adaptation of RNN-Transducer Prediction Network](https://mlsanity.com/paper/2104.11127/) (2021)
- [CTC Variations Through New WFST Topologies](https://mlsanity.com/paper/2110.03098/) (2021)
- [On Architectures and Training for Raw Waveform Feature Extraction in ASR](https://mlsanity.com/paper/2104.04298/) (2021)
- [On lattice-free boosted MMI training of HMM and CTC-based full-context ASR models](https://mlsanity.com/paper/2107.04154/) (2021)
- [Streaming Transformer Transducer Based Speech Recognition Using Non-Causal Convolution](https://mlsanity.com/paper/2110.05241/) (2021)
- [Advancing CTC-CRF Based End-to-End Speech Recognition with Wordpieces and Conformers](https://mlsanity.com/paper/2107.03007/) (2021)
- [Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios](https://mlsanity.com/paper/2104.02232/) (2021)
- [Unsupervised Pre-Training for Vietnamese Automatic Speech Recognition in the HYKIST Project](https://mlsanity.com/paper/2309.15869/) (2023)

## Abstract (arXiv)

In this work, we first show that on the widely used LibriSpeech benchmark, our transformer-based context-dependent connectionist temporal classification (CTC) system produces state-of-the-art results. We then show that using wordpieces as modeling units combined with CTC training, we can greatly simplify the engineering pipeline compared to conventional frame-based cross-entropy training by excluding all the GMM bootstrapping, decision tree building and force alignment steps, while still achieving very competitive word-error-rate. Additionally, using wordpieces as modeling units can significantly improve runtime efficiency since we can use larger stride without losing accuracy. We further confirm these findings on two internal VideoASR datasets: German, which is similar to English as a fusional language, and Turkish, which is an agglutinative language.