# Highway Long Short-Term Memory RNNs for Distant Speech Recognition

arXiv: https://arxiv.org/abs/1510.08983 · HTML: https://mlsanity.com/paper/1510.08983/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2015.

## Built on by (most cited first, 17 of 17)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition](https://mlsanity.com/paper/2105.03842/) (2021)
- [FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition](https://mlsanity.com/paper/2109.14420/) (2021)
- [Imitation Learning for Variable Speed Contact Motion for Operation up to Control Bandwidth](https://mlsanity.com/paper/2102.10283/) (2021)
- [Alignment Knowledge Distillation for Online Streaming Attention-based Speech Recognition](https://mlsanity.com/paper/2103.00422/) (2021)
- [Improving the fusion of acoustic and text representations in RNN-T](https://mlsanity.com/paper/2201.10240/) (2022)
- [Wake Word Detection with Streaming Transformers](https://mlsanity.com/paper/2102.04488/) (2021)
- [Mask the Correct Tokens: An Embarrassingly Simple Approach for Error Correction](https://mlsanity.com/paper/2211.13252/) (2022)
- [Speak While You Think: Streaming Speech Synthesis During Text Generation](https://mlsanity.com/paper/2309.11210/) (2023)
- [BERSting at the Screams: A Benchmark for Distanced, Emotional and Shouted Speech Recognition](https://mlsanity.com/paper/2505.00059/) (2025)
- [Relation Regularized Scene Graph Generation](https://mlsanity.com/paper/2202.10826/) (2022)
- [Syntactic and Semantic-driven Learning for Open Information Extraction](https://mlsanity.com/paper/2103.03448/) (2021)
- [HMM-Free Encoder Pre-Training for Streaming RNN Transducer](https://mlsanity.com/paper/2104.10764/) (2021)
- [VAD-free Streaming Hybrid CTC/Attention ASR for Unsegmented Recording](https://mlsanity.com/paper/2107.07509/) (2021)
- [Efficiency-oriented approaches for self-supervised speech representation learning](https://mlsanity.com/paper/2312.11142/) (2023)
- [Streaming Bilingual End-to-End ASR model using Attention over Multiple Softmax](https://mlsanity.com/paper/2401.11645/) (2024)
- [Bilateral Control-Based Imitation Learning for Velocity-Controlled Robot](https://mlsanity.com/paper/2103.04004/) (2021)

## Abstract (arXiv)

In this paper, we extend the deep long short-term memory (DLSTM) recurrent neural networks by introducing gated direct connections between memory cells in adjacent layers. These direct links, called highway connections, enable unimpeded information flow across different layers and thus alleviate the gradient vanishing problem when building deeper LSTMs. We further introduce the latency-controlled bidirectional LSTMs (BLSTMs) which can exploit the whole history while keeping the latency under control. Efficient algorithms are proposed to train these novel networks using both frame and sequence discriminative criteria. Experiments on the AMI distant speech recognition (DSR) task indicate that we can train deeper LSTMs and achieve better improvement from sequence training with highway LSTMs (HLSTMs). Our novel model obtains 43.9/47.7% WER on AMI (SDM) dev and eval sets, outperforming all previous works. It beats the strong DNN and DLSTM baselines with 15.7% and 5.3% relative improvement respectively.