# Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition

arXiv: https://arxiv.org/abs/1610.09975 · HTML: https://mlsanity.com/paper/1610.09975/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2016.

## Built on by (most cited first, 30 of 33)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [Internal Language Model Training for Domain-Adaptive End-to-End Speech Recognition](https://mlsanity.com/paper/2102.01380/) (2021)
- [Tied & Reduced RNN-T Decoder](https://mlsanity.com/paper/2109.07513/) (2021)
- [MOPRD: A multidisciplinary open peer review dataset](https://mlsanity.com/paper/2212.04972/) (2022)
- [End-to-End Speech Recognition from Federated Acoustic Models](https://mlsanity.com/paper/2104.14297/) (2021)
- [Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages](https://mlsanity.com/paper/2205.01086/) (2022)
- [Google Neural Network Models for Edge Devices: Analyzing and Mitigating Machine Learning Inference Bottlenecks](https://mlsanity.com/paper/2109.14320/) (2021)
- [Accelerating RNN-T Training and Inference Using CTC guidance](https://mlsanity.com/paper/2210.16481/) (2022)
- [Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision](https://mlsanity.com/paper/2406.02166/) (2024)
- [Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languages](https://mlsanity.com/paper/2208.12666/) (2022)
- [Transformer Language Models with LSTM-based Cross-utterance Information Representation](https://mlsanity.com/paper/2102.06474/) (2021)
- [Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification](https://mlsanity.com/paper/2108.02598/) (2021)
- [Hierarchical Conditional End-to-End ASR with CTC and Multi-Granular Subword Units](https://mlsanity.com/paper/2110.04109/) (2021)
- [Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification](https://mlsanity.com/paper/2102.07370/) (2021)
- [Construction of a Large-scale Japanese ASR Corpus on TV Recordings](https://mlsanity.com/paper/2103.14736/) (2021)
- [Have best of both worlds: two-pass hybrid and E2E cascading framework for speech recognition](https://mlsanity.com/paper/2110.04891/) (2021)
- [Exploring the Integration of Large Language Models into Automatic Speech Recognition Systems: An Empirical Study](https://mlsanity.com/paper/2307.06530/) (2023)
- [Match to Win: Analysing Sequences Lengths for Efficient Self-supervised Learning in Speech and Audio](https://mlsanity.com/paper/2209.15575/) (2022)
- [LiteLSTM Architecture for Deep Recurrent Neural Networks](https://mlsanity.com/paper/2201.11624/) (2022)
- [Transformers versus LSTMs for electronic trading](https://mlsanity.com/paper/2309.11400/) (2023)
- [Raw Waveform Encoder with Multi-Scale Globally Attentive Locally Recurrent Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/2106.04275/) (2021)
- [Generalizing RNN-Transducer to Out-Domain Audio via Sparse Self-Attention Layers](https://mlsanity.com/paper/2108.10752/) (2021)
- [Speech recognition for air traffic control via feature learning and end-to-end training](https://mlsanity.com/paper/2111.02654/) (2021)
- [SoftCTC -- Semi-Supervised Learning for Text Recognition using Soft Pseudo-Labels](https://mlsanity.com/paper/2212.02135/) (2022)
- [Dual-Attention Neural Transducers for Efficient Wake Word Spotting in Speech Recognition](https://mlsanity.com/paper/2304.01905/) (2023)
- [Improving Semi-supervised End-to-end Automatic Speech Recognition using CycleGAN and Inter-domain Losses](https://mlsanity.com/paper/2210.11642/) (2022)
- [LiteLSTM Architecture Based on Weights Sharing for Recurrent Neural Networks](https://mlsanity.com/paper/2301.04794/) (2023)
- [Foundations of Multisensory Artificial Intelligence](https://mlsanity.com/paper/2404.18976/) (2024)
- [Low-Resourced Speech Recognition for Iu Mien Language via Weakly-Supervised Phoneme-based Multilingual Pre-training](https://mlsanity.com/paper/2407.13292/) (2024)

## Abstract (arXiv)

We present results that show it is possible to build a competitive, greatly simplified, large vocabulary continuous speech recognition system with whole words as acoustic units. We model the output vocabulary of about 100,000 words directly using deep bi-directional LSTM RNNs with CTC loss. The model is trained on 125,000 hours of semi-supervised acoustic training data, which enables us to alleviate the data sparsity problem for word models. We show that the CTC word models work very well as an end-to-end all-neural speech recognition model without the use of traditional context-dependent sub-word phone units that require a pronunciation lexicon, and without any language model removing the need to decode. We demonstrate that the CTC word models perform better than a strong, more complex, state-of-the-art baseline with sub-word units.