mlsanityData to 7 Oct 2026

Recurrent Neural Network Regularization

arXiv 1409.2329 ↗First posted 8 Sep 2014cs.NEcited by 189

Not on the map. Why?

Abstract (arXiv)

We present a simple regularization technique for Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM) units. Dropout, the most successful technique for regularizing neural networks, does not work well with RNNs and LSTMs. In this paper, we show how to correctly apply dropout to LSTMs, and show that it substantially reduces overfitting on a variety of tasks. These tasks include language modeling, speech recognition, image caption generation, and machine translation.

Key facts. Cited by 189 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Sep 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 2; 2018: 2; 2019: 0; 2020: 0; 2021: 52; 2022: 44; 2023: 25; 2024: 35; 2025: 23; 2026: 6.

189 papers.

Sort:

By area:

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  2. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  3. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  4. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  5. 2023Large Language Models for Robotics: A Surveycited by 132
  6. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  7. 2021Trustworthy AI: A Computational Perspectivecited by 93
  8. 2021CAT: Cross Attention in Vision Transformercited by 92
  9. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
  10. 2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
  11. 2024Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitatecited by 46
  12. 2021Noisy Recurrent Neural Networkscited by 42
38 more
  1. 2022Dissecting Transformer Length Extrapolation via the Lens of Receptive Field Analysiscited by 39
  2. 2024When Large Language Models Meet Vector Databases: A Surveycited by 39
  3. 2022Visual Attention Methods in Deep Learning: An In-Depth Surveycited by 37
  4. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learningcited by 36
  5. 2021AutoDropout: Learning Dropout Patterns to Regularize Deep Networkscited by 35
  6. 2021Recent advances for quantum classifierscited by 35
  7. 2025Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representationscited by 35
  8. 2023Vision-Based Traffic Accident Detection and Anticipation: A Surveycited by 31
  9. 20223D Human Motion Prediction: A Surveycited by 30
  10. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  11. 2021Topology-aware Convolutional Neural Network for Efficient Skeleton-based Action Recognitioncited by 29
  12. 2021Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalizationcited by 28
  13. 2021Attentive Neural Controlled Differential Equations for Time-series Classification and Forecastingcited by 28
  14. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  15. 2021Selfish Sparse RNN Trainingcited by 26
  16. 2022PQLM -- Multilingual Decentralized Portable Quantum Language Model for Privacy Protectioncited by 26
  17. 2022Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcitycited by 24
  18. 2025Pre-training under infinite computecited by 24
  19. 2021Super-resolution-based Change Detection Network with Stacked Attention Module for Images with Different Resolutionscited by 23
  20. 2021Expansion-Squeeze-Excitation Fusion Network for Elderly Activity Recognitioncited by 23
  21. 2022EXIT: Extrapolation and Interpolation-based Neural Controlled Differential Equations for Time-series Classification and Forecastingcited by 22
  22. 2017Reinforced Video Captioning with Entailment Rewardscited by 21
  23. 2022A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directionscited by 21
  24. 2021Learning Connectivity with Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 20
  25. 2022Deep Learning for Computational Cytology: A Surveycited by 20
  26. 2021Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognitioncited by 19
  27. 2021Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrumscited by 15
  28. 2021Cognitive Visual Commonsense Reasoning Using Dynamic Working Memorycited by 14
  29. 2021Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradientscited by 13
  30. 2022Quantum Neural Network Classifiers: A Tutorialcited by 13
  31. 2021An Empirical Study on Hyperparameter Optimization for Fine-Tuning Pre-trained Language Modelscited by 12
  32. 2021Interpretable Visual Understanding with Cognitive Attention Networkcited by 12
  33. 2023Leveraging arbitrary mobile sensor trajectories with shallow recurrent decoder networks for full-state reconstructioncited by 12
  34. 2023Improving Image Captioning via Predicting Structured Conceptscited by 12
  35. 2024HMIL: Hierarchical Multi-Instance Learning for Fine-Grained Whole Slide Image Classificationcited by 12
  36. 2021Brittle AI, Causal Confusion, and Bad Mental Models: Challenges and Successes in the XAI Programcited by 11
  37. 2021EEGDnet: Fusing Non-Local and Local Self-Similarity for 1-D EEG Signal Denoising with 2-D Transformercited by 11
  38. 2025CoAT: Chain-of-Associated-Thoughts Framework for Enhancing Large Language Models Reasoningcited by 11

50 most cited of 189 All 189 (JSON)