# Benchmarking of LSTM Networks

arXiv: https://arxiv.org/abs/1508.02774 · HTML: https://mlsanity.com/paper/1508.02774/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2015.

## Built on by (most cited first, 2 of 2)

- [Prototype of a robotic system to assist the learning process of English language with text-generation through DNN](https://mlsanity.com/paper/2309.11142/) (2023)
- [Deep-Ace: LSTM-based Prokaryotic Lysine Acetylation Site Predictor](https://mlsanity.com/paper/2410.09968/) (2024)

## Abstract (arXiv)

LSTM (Long Short-Term Memory) recurrent neural networks have been highly successful in a number of application areas. This technical report describes the use of the MNIST and UW3 databases for benchmarking LSTM networks and explores the effect of different architectural and hyperparameter choices on performance. Significant findings include: (1) LSTM performance depends smoothly on learning rates, (2) batching and momentum has no significant effect on performance, (3) softmax training outperforms least square training, (4) peephole units are not useful, (5) the standard non-linearities (tanh and sigmoid) perform best, (6) bidirectional training combined with CTC performs better than other methods.