mlsanityData to 7 Oct 2026

Do RNN and LSTM have Long Memory?

arXiv 2006.03860 ↗First posted 6 Jun 2020stat.ML cs.LGcited by 47

Not on the map. Why?

Abstract (arXiv)

The LSTM network was proposed to overcome the difficulty in learning long-term dependence, and has made significant advancements in applications. With its success and drawbacks in mind, this paper raises the question - do RNN and LSTM have long memory? We answer it partially by proving that RNN and LSTM do not have long memory from a statistical perspective. A new definition for long memory networks is further introduced, and it requires the model weights to decay at a polynomial rate. To verify our theory, we convert RNN and LSTM into long memory networks by making a minimal modification, and their superiority is illustrated in modeling long-term dependence of various datasets.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 4; 2023: 13; 2024: 7; 2025: 6; 2026: 7.

47 papers.

Sort:

By area:

  1. 2021CKConv: Continuous Kernel Convolution For Sequential Datacited by 117
  2. 2021Scene-Intuitive Agent for Remote Embodied Visual Groundingcited by 49
  3. 2021Noisy Recurrent Neural Networkscited by 42
  4. 2023LLMR: Real-time Prompting of Interactive Worlds using Large Language Modelscited by 39
  5. 2024Unlocking the Power of LSTM for Long Term Time Series Forecastingcited by 29
  6. 2021A Multi-view Multi-task Learning Framework for Multi-variate Time Series Forecastingcited by 21
  7. 2022A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learningcited by 15
  8. 2021Recurrence along Depth: Deep Convolutional Neural Networks with Recurrent Layer Aggregationcited by 13
  9. 2024Enhancing Automatic Modulation Recognition through Robust Global Feature Extractioncited by 7
  10. 2021On the Memory Mechanism of Tensor-Power Recurrent Modelscited by 5
  11. 2021A Note on Learning Rare Events in Molecular Dynamics using LSTM and Transformercited by 5
  12. 2021Recurrent Neural Networks for Learning Long-term Temporal Dependencies with Reanalysis of Time Scale Representationcited by 4
35 more
  1. 2023Xtal2DoS: Attention-based Crystal to Sequence Learning for Density of States Predictioncited by 4
  2. 2026Deep Generative Spatiotemporal Engression for Probabilistic Forecasting of Epidemicscited by 4
  3. 2022Recurrence-in-Recurrence Networks for Video Deblurringcited by 3
  4. 2022Impartial Games: A Challenge for Reinforcement Learningcited by 3
  5. 2022Don't Watch Me: A Spatio-Temporal Trojan Attack on Deep-Reinforcement-Learning-Augment Autonomous Drivingcited by 3
  6. 2023GAT-GAN : A Graph-Attention-based Time-Series Generative Adversarial Networkcited by 3
  7. 2025UniFault: A Fault Diagnosis Foundation Model from Bearing Datacited by 3
  8. 2023Anamnesic Neural Differential Equations with Orthogonal Polynomial Projectionscited by 2
  9. 2024A Generalized Transformer-based Radio Link Failure Prediction Framework in 5G RANscited by 2
  10. 2024A Foundation Model for Wearable Movement Data in Mental Health Researchcited by 2
  11. 2024IMSSA: Deploying modern state-space models on memristive in-memory compute hardwarecited by 2
  12. 2021Deep Inertial Odometry with Accurate IMU Preintegrationcited by 1
  13. 2021ARISE: ApeRIodic SEmi-parametric Process for Efficient Markets without Periodogram and Gaussianity Assumptionscited by 1
  14. 2023TKN: Transformer-based Keypoint Prediction Network For Real-time Video Predictioncited by 1
  15. 2023Improve Long-term Memory Learning Through Rescaling the Error Temporallycited by 1
  16. 2023Learning Successor Features with Distributed Hebbian Temporal Memorycited by 1
  17. 2023Exploring Parity Challenges in Reinforcement Learning through Curriculum Learning with Noisy Labelscited by 1
  18. 2025Numerical Investigation of Sequence Modeling Theory using Controllable Memory Functionscited by 1
  19. 2025Deep Learning-based Animal Behavior Analysis: Insights from Mouse Chronic Pain Modelscited by 1
  20. 2025Leveraging IndoBERT and DistilBERT for Indonesian Emotion Classification in E-Commerce Reviewscited by 1
  21. 2023Extending Memory for Language Modellingcited by 0
  22. 2023On the Role of Noise in the Sample Complexity of Learning Recurrent Neural Networks: Exponential Gaps for Long Sequencescited by 0
  23. 2023Recurrent Neural Networks with more flexible memory: better predictions than rough volatilitycited by 0
  24. 2023SurvTimeSurvival: Survival Analysis On The Patient With Multiple Visits/Recordscited by 0
  25. 2023Curriculum Design Helps Spiking Neural Networks to Classify Time Seriescited by 0
  26. 2024Temporal receptive field in dynamic graph learning: A comprehensive analysiscited by 0
  27. 2024Making Every Frame Matter: Continuous Activity Recognition in Streaming Video via Adaptive Video Context Modelingcited by 0
  28. 2025Feature-Enhanced Machine Learning for All-Cause Mortality Prediction in Healthcare Datacited by 0
  29. 2025Neural ARFIMA model for forecasting BRIC exchange rates with long memorycited by 0
  30. 2026Generative Artificial Intelligence for Literature Reviewscited by 0
  31. 2026Long-Memory Reservoir Computing for Data-Scarce Dengue Forecastingcited by 0
  32. 2026An Auto-Scaling Approach for Serverless Environments Based on a Multi-Expert Consensus Mechanismcited by 0
  33. 2026Identifying parameter couplings and uncertainties of mixed-noise stochastic systems via full-covariance Gaussian mixture networkcited by 0
  34. 2026SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecastingcited by 0
  35. 2026Valid and Efficient Split Conformal Regression for Time Seriescited by 0