mlsanityData to 7 Oct 2026

Recurrent Neural Networks and Long Short-Term Memory Networks: Tutorial and Survey

arXiv 2304.11461 ↗First posted 22 Apr 2023cs.LG cs.CL cs.NEcited by 10

Not on the map. Why?

Abstract (arXiv)

This is a tutorial paper on Recurrent Neural Network (RNN), Long Short-Term Memory Network (LSTM), and their variants. We start with a dynamical system and backpropagation through time for RNN. Then, we discuss the problems of gradient vanishing and explosion in long-term dependencies. We explain close-to-identity weight matrix, long delays, leaky units, and echo state networks for solving this problem. Then, we introduce LSTM gates and cells, history and variants of LSTM, and Gated Recurrent Units (GRU). Finally, we introduce bidirectional RNN, bidirectional LSTM, and the Embeddings from Language Model (ELMo) network, for processing a sequence in both directions.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 18 of its 59 references that are in the map. First posted on arXiv 22 Apr 2023.

Builds on

18 of 59 references.

References · 18

  1. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  2. 2018Deep contextualized word representationscited by 2,285
  3. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  4. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  5. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  6. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  7. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 544
  8. 2015LSTM: A Search Space Odysseycited by 315
  9. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  10. 2015A Critical Review of Recurrent Neural Networks for Sequence Learningcited by 223
  11. 2019Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networkscited by 103
  12. 2017Recent Advances in Recurrent Neural Networkscited by 89
6 more
  1. 2017Semi-supervised sequence tagging with bidirectional language modelscited by 86
  2. 2019Eigenvalue and Generalized Eigenvalue Problems: Tutorialcited by 45
  3. 2021KKT Conditions, First-Order and Second-Order Optimization, and Distributed Optimization: Tutorial and Surveycited by 14
  4. 2017Simplified Minimal Gated Unit Variations for Recurrent Neural Networkscited by 13
  5. 2014Learning Longer Memory in Recurrent Neural Networkscited by 11
  6. 2018Overcoming the vanishing gradient problem in plain recurrent networkscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 2; 2026: 4.

10 papers.

  1. 2024Human-inspired Perspectives: A Survey on AI Long-term Memorycited by 12
  2. 2024Recurrent Stochastic Configuration Networks for Temporal Data Analyticscited by 7
  3. 2023Physics-based Reduced Order Modeling for Uncertainty Quantification of Guided Wave Propagation using Bayesian Optimizationcited by 1
  4. 2026Predicting Post-Traumatic Epilepsy from Clinical Records using Large Language Model Embeddingscited by 1
  5. 2024Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modellingcited by 0
  6. 2025Can Modern NLP Systems Reliably Annotate Chest Radiography Exams? A Pre-Purchase Evaluation and Comparative Study of Solutions from AWS, Google, Azure, John Snow Labs, and Open-Source Models on an Independent Pediatric Datasetcited by 0
  7. 2025CNN-LSTM Hybrid Model for AI-Driven Prediction of COVID-19 Severity from Spike Sequences and Clinical Datacited by 0
  8. 2026Topological Neural Dynamics: A Neuron-wise Framework for Sequence Modelingcited by 0
  9. 2026Lightweight Safe Reinforcement Learning for End-to-End UAV Navigationcited by 0
  10. 2026MTLiquid: Enabling Efficient Multi-Task Learning using Liquid Neural Networks for Lightweight Healthcare Monitoring Systemscited by 0