mlsanityData to 7 Oct 2026

AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks

arXiv 1902.09689 ↗First posted 26 Feb 2019stat.ML cs.LGcited by 112

Not on the map. Why?

Abstract (arXiv)

Recurrent neural networks have gained widespread use in modeling sequential data. Learning long-term dependencies using these models remains difficult though, due to exploding or vanishing gradients. In this paper, we draw connections between recurrent networks and ordinary differential equations. A special form of recurrent networks called the AntisymmetricRNN is proposed under this theoretical framework, which is able to capture long-term dependencies thanks to the stability property of its underlying differential equation. Existing approaches to improving RNN trainability often incur significant computation overhead. In comparison, AntisymmetricRNN achieves the same goal by design. We showcase the advantage of this new architecture through extensive simulations and experiments. AntisymmetricRNN exhibits much more predictable dynamics. It outperforms regular LSTM models on tasks requiring long-term memory and matches the performance on tasks where short-term dependencies dominate despite being much simpler.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 24; 2023: 22; 2024: 15; 2025: 16; 2026: 10.

112 papers.

Sort:

By area:

  1. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  2. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  3. 2021Efficient and Accurate Gradients for Neural SDEscited by 68
  4. 2021FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizescited by 66
  5. 2021Closed-form Continuous-time Neural Modelscited by 65
  6. 2021On the difficulty of learning chaotic dynamics with RNNscited by 64
  7. 2022Anti-Symmetric DGN: a stable architecture for Deep Graph Networkscited by 58
  8. 2021UnICORNN: A recurrent model for learning very long time dependenciescited by 56
  9. 2023Generalized Teacher Forcing for Learning Chaotic Dynamicscited by 55
  10. 2021Neural Controlled Differential Equations for Online Prediction Taskscited by 46
  11. 2021Noisy Recurrent Neural Networkscited by 42
  12. 2021Long Expressive Memory for Sequence Modelingcited by 41
38 more
  1. 2021Understanding Object Dynamics for Interactive Image-to-Video Synthesiscited by 38
  2. 2021A Framework for Machine Learning of Model Error in Dynamical Systemscited by 37
  3. 2022Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systemscited by 34
  4. 2023Physics-Informed Machine Learning for Modeling and Control of Dynamical Systemscited by 28
  5. 2022projUNN: efficient method for training deep networks with unitary matricescited by 27
  6. 2021Neural Delay Differential Equationscited by 26
  7. 2022Efficient recurrent architectures through activity sparsity and sparse back-propagation through timecited by 26
  8. 2021Framing RNN as a kernel method: A neural ODE approachcited by 23
  9. 2021Hamiltonian Deep Neural Networks Guaranteeing Non-vanishing Gradients by Designcited by 18
  10. 2022Scaling ResNets in the Large-depth Regimecited by 18
  11. 2023Robustifying State-space Models for Long Sequences via Approximate Diagonalizationcited by 18
  12. 2023Traveling Waves Encode the Recent Past and Enhance Sequence Learningcited by 16
  13. 2024Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patchingcited by 16
  14. 2024Tuning Frequency Bias of State Space Modelscited by 16
  15. 2024Temporally Consistent Koopman Autoencoders for Forecasting Dynamical Systemscited by 15
  16. 2024DyG-Mamba: Continuous State Space Modeling on Dynamic Graphscited by 15
  17. 2021Stateful ODE-Nets using Basis Function Expansionscited by 14
  18. 2022Homotopy-based training of NeuralODEs for accurate dynamics discoverycited by 14
  19. 2022Robust Classification using Contractive Hamiltonian Neural ODEscited by 13
  20. 2025Return of ChebNet: Understanding and Improving an Overlooked GNN on Long Range Taskscited by 12
  21. 2022Flipped Classroom: Effective Teaching for Time Series Forecastingcited by 11
  22. 2024MedDet: Generative Adversarial Distillation for Efficient Cervical Disc Herniation Detectioncited by 11
  23. 2022Dynamical systems' based neural networkscited by 10
  24. 2021Accelerating Neural ODEs Using Model Order Reductioncited by 9
  25. 2023Neural oscillators for generalization of physics-informed machine learningcited by 9
  26. 2022Deep equilibrium networks are sensitive to initialization statisticscited by 8
  27. 2021RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networkscited by 7
  28. 2023Controllable Video Generation by Learning the Underlying Dynamical System with Neural ODEcited by 7
  29. 2025Block-Biased Mamba for Long-Range Sequence Processingcited by 7
  30. 2023Persistent learning signals and working memory without continuous attractorscited by 6
  31. 2025Over-squashing in Spatiotemporal Graph Neural Networkscited by 6
  32. 2022Parallel Training of GRU Networks with a Multi-Grid Solver for Long Sequencescited by 5
  33. 2022Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networkscited by 5
  34. 2023Gradient Flossing: Improving Gradient Descent through Dynamic Control of Jacobianscited by 5
  35. 2021A unified framework for Hamiltonian deep neural networkscited by 4
  36. 2022Neural Piecewise-Constant Delay Differential Equationscited by 4
  37. 2022Stable Invariant Models via Koopman Spectracited by 4
  38. 2023Neural Differential Recurrent Neural Network with Adaptive Time Stepscited by 4

50 most cited of 112 All 112 (JSON)