Data to 7 Oct 2026

Stable Recurrent Models

arXiv 1805.10369 ↗First posted 25 May 2018cs.LG stat.MLcited by 65

Not on the map. Why?

Abstract (arXiv)

Stability is a fundamental property of dynamical systems, yet to this date it has had little bearing on the practice of recurrent neural networks. In this work, we conduct a thorough investigation of stable recurrent models. Theoretically, we prove stable recurrent neural networks are well approximated by feed-forward networks for the purpose of both inference and training by gradient descent. Empirically, we demonstrate stable recurrent models often perform as well as their unstable counterparts on benchmark sequence tasks. Taken together, these findings shed light on the effective power of recurrent networks and suggest much of sequence learning happens, or can be made to happen, in the stable regime. Moreover, our results help to explain why in many cases practitioners succeed in replacing recurrent models by feed-forward models.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 7; 2023: 13; 2024: 7; 2025: 8; 2026: 12.

65 papers.

Sort:

By area:

2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 251
2021Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustnesscited by 43
2021Noisy Recurrent Neural Networkscited by 42
2023Deep networks for system identification: a Surveycited by 35
2022A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biasescited by 30
2022Fine-Tuning Pre-trained Transformers into Decaying Fast Weightscited by 26
2021DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programscited by 19
2021Near-optimal Offline and Streaming Algorithms for Learning Non-Linear Dynamical Systemscited by 17
2021Almost Surely Stable Deep Dynamicscited by 16
2022DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programscited by 16
2024Temporally Consistent Koopman Autoencoders for Forecasting Dynamical Systemscited by 15

38 more

2022Homotopy-based training of NeuralODEs for accurate dynamics discoverycited by 14
2021Learning over All Stabilizing Nonlinear Controllers for a Partially-Observed Linear Systemcited by 13
2021Convergent Graph Solverscited by 11
2025An Optimisation Framework for Unsupervised Environment Designcited by 9
2021RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networkscited by 7
2024Learning to optimize with convergence guarantees using nonlinear system theorycited by 7
2025Predictability Enables Parallelization of Nonlinear State Space Modelscited by 7
2021Stable Long-Term Recurrent Video Super-Resolutioncited by 6
2023Learning Over Contracting and Lipschitz Closed-Loops for Partially-Observed Nonlinear Systems (Extended Version)cited by 6
2023Adaptive recurrent vision performs zero-shot computation scaling to unseen difficulty levelscited by 6
2021On the Memory Mechanism of Tensor-Power Recurrent Modelscited by 5
2021Recurrent Neural Network-based Internal Model Control design for stable nonlinear systemscited by 5
2023Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalabilitycited by 5
2026Pretraining Recurrent Networks without Recurrencecited by 5
2021Memory and attention in deep learningcited by 4
2023SDYN-GANs: Adversarial Learning Methods for Multistep Generative Models for General Order Stochastic Dynamicscited by 4
2023Structured state-space models are deep Wiener modelscited by 4
2024Fuzzy Recurrent Stochastic Configuration Networks for Industrial Data Analyticscited by 4
2021A Differential Geometry Perspective on Orthogonal Recurrent Modelscited by 3
2021Robust Learning of Recurrent Neural Networks in Presence of Exogenous Noisecited by 3
2022On Scrambling Phenomena for Randomly Initialized Recurrent Networkscited by 3
2024Distributed Representations Enable Robust Multi-Timescale Symbolic Computation in Neuromorphic Hardwarecited by 3
2026Koopman Dreamer: Spectrally Constrained Latent Dynamics for Stable World-Model Imaginationcited by 3
2022Robustness Analysis of Classification Using Recurrent Neural Networks with Perturbed Sequential Inputcited by 2
2023Fading memory as inductive bias in residual recurrent networkscited by 2
2025Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connectionscited by 2
2025Fast attention mechanisms: a tale of parallelismcited by 2
2021Accelerating Dynamical System Simulations with Contracting and Physics-Projected Neural-Newton Solverscited by 1
2022Reinforcement Learning and Tree Search Methods for the Unit Commitment Problemcited by 1
2023Learning Stable and Robust Linear Parameter-Varying State-Space Modelscited by 1
2023Behavior quantification as the missing link between fields: Tools for digital psychiatry and their role in the future of neurobiologycited by 1
2023Learning Hybrid Dynamics Models With Simulator-Informed Latent Statescited by 1
2023DREAM: Debugging and Repairing AutoML Pipelinescited by 1
2025Latent Trajectory Dynamics in Large Language Models: A Manifold Evolution Framework with Empirical Validationcited by 1
2025Numerical Investigation of Sequence Modeling Theory using Controllable Memory Functionscited by 1
2026Tuning the burn-in phase in training recurrent neural networks improves their performancecited by 1
2026Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filteringcited by 1
2026Looped Transformers with Source-Centered State Evolutioncited by 1

50 most cited of 65 All 65 (JSON)