A recurrent neural network without chaos
Not on the map. Why?
Abstract (arXiv)
We introduce an exceptionally simple gated recurrent neural network (RNN) that achieves performance comparable to well-known gated architectures, such as LSTMs and GRUs, on the word-level language modeling task. We prove that our model has simple, predicable and non-chaotic dynamics. This stands in stark contrast to more standard gated architectures, whose underlying dynamical systems exhibit chaotic behavior.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Dec 2016.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 3; 2023: 2; 2024: 2; 2025: 2; 2026: 1.
13 papers.
Sort:
By area:
- 2021UnICORNN: A recurrent model for learning very long time dependenciescited by 56
- 2022Fine-Tuning Pre-trained Transformers into Decaying Fast Weightscited by 26
- 2023Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environmentscited by 5
- 2021Recurrent Neural Networks for Learning Long-term Temporal Dependencies with Reanalysis of Time Scale Representationcited by 4
- 2022A Convergence Analysis of Nesterov's Accelerated Gradient Method in Training Deep Linear Neural Networkscited by 4
- 2021Online Stochastic Gradient Descent Learns Linear Dynamical Systems from A Single Trajectorycited by 3
- 2022On Scrambling Phenomena for Randomly Initialized Recurrent Networkscited by 3
- 2024Mamba State-Space Models Are Lyapunov-Stable Learnerscited by 3
- 2024Exploiting Chaotic Dynamics as Deep Neural Networkscited by 2
- 2023Impression-Aware Recommender Systemscited by 1
- 2025Hyperpruning: Efficient Search through Pruned Variants of Recurrent Neural Networks Leveraging Lyapunov Spectrumcited by 1
- 2025Algorithm Development in Neural Networks: Insights from the Streaming Parity Taskcited by 0
- 2026Divisive Normalization Shapes Low-Rank Slow Manifolds for Continuous Working Memory3 Aug 2026
- 2025Algorithm Development in Neural Networks: Insights from the Streaming Parity Task14 Jul 2025
- 2025Hyperpruning: Efficient Search through Pruned Variants of Recurrent Neural Networks Leveraging Lyapunov Spectrum9 Jun 2025
- 2024Mamba State-Space Models Are Lyapunov-Stable Learners31 May 2024
- 2024Exploiting Chaotic Dynamics as Deep Neural Networks29 May 2024
- 2023Impression-Aware Recommender Systems15 Aug 2023
- 2023Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environments14 Apr 2023
- 2022On Scrambling Phenomena for Randomly Initialized Recurrent Networks11 Oct 2022
- 2022Fine-Tuning Pre-trained Transformers into Decaying Fast Weights9 Oct 2022
- 2022A Convergence Analysis of Nesterov's Accelerated Gradient Method in Training Deep Linear Neural Networks18 Apr 2022
- 2021Recurrent Neural Networks for Learning Long-term Temporal Dependencies with Reanalysis of Time Scale Representation5 Nov 2021
- 2021UnICORNN: A recurrent model for learning very long time dependencies9 Mar 2021
- 2024Mamba State-Space Models Are Lyapunov-Stable Learnerscited by 3
- 2024Mamba State-Space Models Are Lyapunov-Stable Learners31 May 2024
- 2021UnICORNN: A recurrent model for learning very long time dependenciescited by 56
- 2022Fine-Tuning Pre-trained Transformers into Decaying Fast Weightscited by 26
- 2023Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environmentscited by 5
- 2021Recurrent Neural Networks for Learning Long-term Temporal Dependencies with Reanalysis of Time Scale Representationcited by 4
- 2022A Convergence Analysis of Nesterov's Accelerated Gradient Method in Training Deep Linear Neural Networkscited by 4
- 2021Online Stochastic Gradient Descent Learns Linear Dynamical Systems from A Single Trajectorycited by 3
- 2022On Scrambling Phenomena for Randomly Initialized Recurrent Networkscited by 3
- 2024Exploiting Chaotic Dynamics as Deep Neural Networkscited by 2
- 2023Impression-Aware Recommender Systemscited by 1
- 2025Hyperpruning: Efficient Search through Pruned Variants of Recurrent Neural Networks Leveraging Lyapunov Spectrumcited by 1
- 2025Algorithm Development in Neural Networks: Insights from the Streaming Parity Taskcited by 0
- 2026Divisive Normalization Shapes Low-Rank Slow Manifolds for Continuous Working Memorycited by 0
- 2026Divisive Normalization Shapes Low-Rank Slow Manifolds for Continuous Working Memory3 Aug 2026
- 2025Algorithm Development in Neural Networks: Insights from the Streaming Parity Task14 Jul 2025
- 2025Hyperpruning: Efficient Search through Pruned Variants of Recurrent Neural Networks Leveraging Lyapunov Spectrum9 Jun 2025
- 2024Exploiting Chaotic Dynamics as Deep Neural Networks29 May 2024
- 2023Impression-Aware Recommender Systems15 Aug 2023
- 2023Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environments14 Apr 2023
- 2022On Scrambling Phenomena for Randomly Initialized Recurrent Networks11 Oct 2022
- 2022Fine-Tuning Pre-trained Transformers into Decaying Fast Weights9 Oct 2022
- 2022A Convergence Analysis of Nesterov's Accelerated Gradient Method in Training Deep Linear Neural Networks18 Apr 2022
- 2021Recurrent Neural Networks for Learning Long-term Temporal Dependencies with Reanalysis of Time Scale Representation5 Nov 2021
- 2021UnICORNN: A recurrent model for learning very long time dependencies9 Mar 2021
- 2021Online Stochastic Gradient Descent Learns Linear Dynamical Systems from A Single Trajectory23 Feb 2021