# Constructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech Recognition

arXiv: https://arxiv.org/abs/1410.4281 · HTML: https://mlsanity.com/paper/1410.4281/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2014.

## Built on by (most cited first, 12 of 12)

- [An overview of mixing augmentation methods and augmentation strategies](https://mlsanity.com/paper/2107.09887/) (2021)
- [Improved Training for End-to-End Streaming Automatic Speech Recognition Model with Punctuation](https://mlsanity.com/paper/2306.01296/) (2023)
- [Automatic speech recognition for the Nepali language using CNN, bidirectional LSTM and ResNet](https://mlsanity.com/paper/2406.17825/) (2024)
- [A singular Riemannian Geometry Approach to Deep Neural Networks III. Piecewise Differentiable Layers and Random Walks on n-dimensional Classes](https://mlsanity.com/paper/2404.06104/) (2024)
- [On the Privacy Risks of Deploying Recurrent Neural Networks in Machine Learning Models](https://mlsanity.com/paper/2110.03054/) (2021)
- [DS-TDNN: Dual-stream Time-delay Neural Network with Global-aware Filter for Speaker Verification](https://mlsanity.com/paper/2303.11020/) (2023)
- [Implementation of Artificial Neural Networks for the Nepta-Uranian Interplanetary (NUIP) Mission](https://mlsanity.com/paper/2103.11843/) (2021)
- [Matching Writers to Content Writing Tasks](https://mlsanity.com/paper/2204.09718/) (2022)
- [Compromised account detection using authorship verification: a novel approach](https://mlsanity.com/paper/2206.03581/) (2022)
- [Design Perspectives of Multitask Deep Learning Models and Applications](https://mlsanity.com/paper/2209.13444/) (2022)
- [Learning Knowledge-Rich Sequential Model for Planar Homography Estimation in Aerial Video](https://mlsanity.com/paper/2304.02715/) (2023)
- [An inclusive review on deep learning techniques and their scope in handwriting recognition](https://mlsanity.com/paper/2404.08011/) (2024)

## Abstract (arXiv)

Long short-term memory (LSTM) based acoustic modeling methods have recently been shown to give state-of-the-art performance on some speech recognition tasks. To achieve a further performance improvement, in this research, deep extensions on LSTM are investigated considering that deep hierarchical model has turned out to be more efficient than a shallow one. Motivated by previous research on constructing deep recurrent neural networks (RNNs), alternative deep LSTM architectures are proposed and empirically evaluated on a large vocabulary conversational telephone speech recognition task. Meanwhile, regarding to multi-GPU devices, the training process for LSTM networks is introduced and discussed. Experimental results demonstrate that the deep LSTM networks benefit from the depth and yield the state-of-the-art performance on this task.