mlsanityData to 14 Oct 2026

Can recurrent neural networks warp time?

arXiv 1804.11188 ↗First posted 23 Mar 2018cs.LG cs.NE stat.MLcited by 55

Not on the map. Why?

Abstract (arXiv)

Successful recurrent models such as long short-term memories (LSTMs) and gated recurrent units (GRUs) use ad hoc gating mechanisms. Empirically these models have been found to improve the learning of medium to long term temporal dependencies and to help with vanishing gradient issues. We prove that learnable gates in a recurrent model formally provide quasi- invariance to general time transformations in the input data. We recover part of the LSTM architecture from a simple axiomatic approach. This result leads to a new way of initializing gate biases in LSTMs and GRUs. Ex- perimentally, this new chrono initialization is shown to greatly improve learning of long term dependencies, with minimal implementation effort.

Key facts. Cited by 55 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 7; 2023: 11; 2024: 15; 2025: 8; 2026: 5.

55 papers.

Sort:

By area:

  1. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  2. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 608
  3. 2023Hierarchically Gated Recurrent Neural Network for Sequence Modelingcited by 102
  4. 2024Theoretical Foundations of Deep Selective State-Space Modelscited by 74
  5. 2024A Survey on Visual Mambacited by 50
  6. 2021Long Expressive Memory for Sequence Modelingcited by 41
  7. 2024Mamba-R: Vision Mamba ALSO Needs Registerscited by 41
  8. 2024State-space models are accurate and efficient neural operators for dynamical systemscited by 28
  9. 2021Noether Networks: Meta-Learning Useful Conserved Quantitiescited by 26
  10. 2024Recurrent neural networks: vanishing and exploding gradients are not the end of the storycited by 25
  11. 2024P-Mamba: Marrying Perona Malik Diffusion with Mamba for Efficient Pediatric Echocardiographic Left Ventricular Segmentationcited by 21
  12. 2023Traveling Waves Encode the Recent Past and Enhance Sequence Learningcited by 16
38 more
  1. 2024Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patchingcited by 16
  2. 2023Evaluating the Robustness of Interpretability Methods through Explanation Invariance and Equivariancecited by 14
  3. 2022Learnable Commutative Monoids for Graph Neural Networkscited by 13
  4. 2024Adventurer: Optimizing Vision Mamba Architecture Designs for Efficiencycited by 9
  5. 2024Chimera: Effectively Modeling Multivariate Time Series with 2-Dimensional State Space Modelscited by 8
  6. 2021Slower is Better: Revisiting the Forgetting Mechanism in LSTM for Slower Information Decaycited by 7
  7. 2023Feature Programming for Multivariate Time Series Predictioncited by 7
  8. 2022Gated Recurrent Neural Networks with Weighted Time-Delay Feedbackcited by 5
  9. 2023Time-Warping Invariant Quantum Recurrent Neural Networks via Quantum-Classical Adaptive Gatingcited by 5
  10. 2023The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Taskscited by 5
  11. 2021Recurrent Neural Networks for Learning Long-term Temporal Dependencies with Reanalysis of Time Scale Representationcited by 4
  12. 2022Path sampling of recurrent neural networks by incorporating known physicscited by 4
  13. 2023Probable Object Location (POLo) Score Estimation for Efficient Object Goal Navigationcited by 4
  14. 2024FACTS: A Factored State-Space Framework For World Modellingcited by 4
  15. 2025Monocular visual simultaneous localization and mapping: (r)evolution from geometry to deep learning-based pipelinescited by 4
  16. 2021Online Stochastic Gradient Descent Learns Linear Dynamical Systems from A Single Trajectorycited by 3
  17. 2021Warming up recurrent neural networks to maximise reachable multistability greatly improves learningcited by 3
  18. 2023Inductive biases in deep learning models for weather predictioncited by 3
  19. 2024Neural timescales from a computational perspectivecited by 3
  20. 2021Attention to Warp: Deep Metric Learning for Multivariate Time Seriescited by 2
  21. 2023Optimum Output Long Short-Term Memory Cell for High-Frequency Trading Forecastingcited by 2
  22. 2024Modular Growth of Hierarchical Networks: Efficient, General, and Robust Curriculum Learningcited by 2
  23. 2025Time-Scale Coupling Between States and Parameters in Recurrent Neural Networkscited by 2
  24. 2022Task-Synchronized Recurrent Neural Networkscited by 1
  25. 2022A Probabilistic Interpretation of Transformerscited by 1
  26. 2023Transitions in echo index and dependence on input repetitionscited by 1
  27. 2024Quantized Approximately Orthogonal Recurrent Neural Networkscited by 1
  28. 2025Rethinking Time Encoding via Learnable Transformation Functionscited by 1
  29. 2025RNN as Linear Transformer: A Closer Investigation into Representational Potentials of Visual Mamba Modelscited by 1
  30. 2025KOSS: Kalman-Optimal Selective State Spaces for Long-Term Sequence Modelingcited by 1
  31. 2026Capture Timing-Attention of Events in Clinical Time Seriescited by 1
  32. 2022Fast Saturating Gate for Learning Long Time Scales with Recurrent Neural Networkscited by 0
  33. 2022Circling Back to Recurrent Models of Languagecited by 0
  34. 2023Joint Alignment of Multivariate Quasi-Periodic Functional Data Using Deep Learningcited by 0
  35. 2024Spacetime E(n)-Transformer: Equivariant Attention for Spatio-temporal Graphscited by 0
  36. 2024Stochastic Layer-Wise Shuffle for Improving Vision Mamba Trainingcited by 0
  37. 2025On the dimension of pullback attractors in recurrent neural networkscited by 0
  38. 2025Learnability Window in Gated Recurrent Neural Networkscited by 0

50 most cited of 55 All 55 (JSON)