mlsanityData to 7 Oct 2026

Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies

arXiv 2010.00951 ↗First posted 2 Oct 2020cs.LG cs.NE stat.MLcited by 81

Not on the map. Why?

Abstract (arXiv)

Circuits of biological neurons, such as in the functional parts of the brain can be modeled as networks of coupled oscillators. Inspired by the ability of these systems to express a rich set of outputs while keeping (gradients of) state variables bounded, we propose a novel architecture for recurrent neural networks. Our proposed RNN is based on a time-discretization of a system of second-order ordinary differential equations, modeling networks of controlled nonlinear oscillators. We prove precise bounds on the gradients of the hidden states, leading to the mitigation of the exploding and vanishing gradient problem for this RNN. Experiments show that the proposed RNN is comparable in performance to the state of the art on a variety of benchmarks, demonstrating the potential of this architecture to provide stable and accurate RNNs for processing complex sequential data.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 9; 2023: 18; 2024: 11; 2025: 15; 2026: 15.

81 papers.

Sort:

By area:

  1. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  2. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 327
  3. 2023On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topologycited by 163
  4. 2022Liquid Structural State-Space Modelscited by 134
  5. 2022Graph-Coupled Oscillator Networkscited by 123
  6. 2021FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizescited by 66
  7. 2021Closed-form Continuous-time Neural Modelscited by 65
  8. 2021On the difficulty of learning chaotic dynamics with RNNscited by 64
  9. 2021UnICORNN: A recurrent model for learning very long time dependenciescited by 56
  10. 2023Generalized Teacher Forcing for Learning Chaotic Dynamicscited by 55
  11. 2023Locality-Aware Graph-Rewiring in GNNscited by 55
  12. 2021Noisy Recurrent Neural Networkscited by 42
38 more
  1. 2025On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learningcited by 42
  2. 2021Long Expressive Memory for Sequence Modelingcited by 41
  3. 2024Artificial Kuramoto Oscillatory Neuronscited by 37
  4. 2021Momentum Residual Neural Networkscited by 36
  5. 2024Oscillatory State-Space Modelscited by 35
  6. 2022Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systemscited by 34
  7. 2023StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterizationcited by 29
  8. 2022Efficient recurrent architectures through activity sparsity and sparse back-propagation through timecited by 26
  9. 2024Recurrent neural networks: vanishing and exploding gradients are not the end of the storycited by 25
  10. 2021Causal Navigation by Continuous-time Neural Networkscited by 23
  11. 2023Neural Oscillators are Universalcited by 16
  12. 2023Traveling Waves Encode the Recent Past and Enhance Sequence Learningcited by 16
  13. 2024Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patchingcited by 16
  14. 2021Stateful ODE-Nets using Basis Function Expansionscited by 14
  15. 2022Flipped Classroom: Effective Teaching for Time Series Forecastingcited by 11
  16. 2023Neural oscillators for generalization of physics-informed machine learningcited by 9
  17. 2021Slower is Better: Revisiting the Forgetting Mechanism in LSTM for Slower Information Decaycited by 7
  18. 2024Input-to-State Stable Coupled Oscillator Networks for Closed-form Model-based Control in Latent Spacecited by 7
  19. 2025Learning to Dissipate Energy in Oscillatory State-Space Modelscited by 7
  20. 2023SMPConv: Self-moving Point Representations for Continuous Convolutioncited by 6
  21. 2021DeepSITH: Efficient Learning via Decomposition of What and When Across Time Scalescited by 5
  22. 2022Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networkscited by 5
  23. 2022Gated Recurrent Neural Networks with Weighted Time-Delay Feedbackcited by 5
  24. 2023Gradient Flossing: Improving Gradient Descent through Dynamic Control of Jacobianscited by 5
  25. 2022Sequence Learning Using Equilibrium Propagationcited by 4
  26. 2023Neural Differential Recurrent Neural Network with Adaptive Time Stepscited by 4
  27. 2023Delayed Memory Unit: Modelling Temporal Dependency Through Delay Gatecited by 4
  28. 2024Unconditional stability of a recurrent neural circuit implementing divisive normalizationcited by 4
  29. 2024Generative Modeling of Neural Dynamics via Latent Stochastic Differential Equationscited by 4
  30. 2025Kuramoto Orientation Diffusion Modelscited by 4
  31. 2022Lyapunov-Guided Representation of Recurrent Neural Network Performancecited by 3
  32. 2024Subhomogeneous Deep Equilibrium Modelscited by 3
  33. 2024Deep Loss Convexification for Learning Iterative Modelscited by 3
  34. 2025Detecting Invariant Manifolds in ReLU-Based RNNscited by 3
  35. 2025Stuart-Landau Oscillatory Graph Neural Networkcited by 3
  36. 2021Oscillatory Fourier Neural Network: A Compact and Efficient Architecture for Sequential Processingcited by 2
  37. 2023Trainability, Expressivity and Interpretability in Gated Neural ODEscited by 2
  38. 2023Fading memory as inductive bias in residual recurrent networkscited by 2

50 most cited of 81 All 81 (JSON)