# A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1907.02649 · HTML: https://mlsanity.com/paper/1907.02649/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2019.

## Built on by (most cited first, 30 of 38)

- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies](https://mlsanity.com/paper/2112.13835/) (2021)
- [Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation](https://mlsanity.com/paper/2205.03076/) (2022)
- [Online learning of long-range dependencies](https://mlsanity.com/paper/2305.15947/) (2023)
- [Online Variational Filtering and Parameter Learning](https://mlsanity.com/paper/2110.13549/) (2021)
- [Latent State Models of Training Dynamics](https://mlsanity.com/paper/2308.09543/) (2023)
- [How connectivity structure shapes rich and lazy learning in neural circuits](https://mlsanity.com/paper/2310.08513/) (2023)
- [How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective](https://mlsanity.com/paper/2106.08453/) (2021)
- [Uncovering the Representation of Spiking Neural Networks Trained with Surrogate Gradient](https://mlsanity.com/paper/2304.13098/) (2023)
- [Backpropagation through space, time, and the brain](https://mlsanity.com/paper/2403.16933/) (2024)
- [Real-Time Recurrent Learning using Trace Units in Reinforcement Learning](https://mlsanity.com/paper/2409.01449/) (2024)
- [Persistent learning signals and working memory without continuous attractors](https://mlsanity.com/paper/2308.12585/) (2023)
- [Real-Time Recurrent Reinforcement Learning](https://mlsanity.com/paper/2311.04830/) (2023)
- [SparseProp: Efficient Event-Based Simulation and Training of Sparse Recurrent Spiking Neural Networks](https://mlsanity.com/paper/2312.17216/) (2023)
- [Distinguishing Learning Rules with Brain Machine Interfaces](https://mlsanity.com/paper/2206.13448/) (2022)
- [Gradient Flossing: Improving Gradient Descent through Dynamic Control of Jacobians](https://mlsanity.com/paper/2312.17306/) (2023)
- [Online hyperparameter optimization by real-time recurrent learning](https://mlsanity.com/paper/2102.07813/) (2021)
- [Prediction of the Position of External Markers Using a Recurrent Neural Network Trained With Unbiased Online Recurrent Optimization for Safe Lung Cancer Radiotherapy](https://mlsanity.com/paper/2106.01100/) (2021)
- [Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules](https://mlsanity.com/paper/2206.00823/) (2022)
- [Efficient Real Time Recurrent Learning through combined activity and parameter sparsity](https://mlsanity.com/paper/2303.05641/) (2023)
- [LongSSM: On the Length Extension of State-space Models in Language Modelling](https://mlsanity.com/paper/2406.02080/) (2024)
- [Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learning](https://mlsanity.com/paper/2503.21419/) (2025)
- [Current State and Future Directions for Learning in Biological Recurrent Neural Networks: A Perspective Piece](https://mlsanity.com/paper/2105.05382/) (2021)
- [Online Evolutionary Neural Architecture Search for Multivariate Non-Stationary Time Series Forecasting](https://mlsanity.com/paper/2302.10347/) (2023)
- [Real-time respiratory motion forecasting with online learning of recurrent neural networks for accurate targeting in externally guided radiotherapy](https://mlsanity.com/paper/2403.01607/) (2024)
- [A Unified Framework for Neural Computation and Learning Over Time](https://mlsanity.com/paper/2409.12038/) (2024)
- [Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Example](https://mlsanity.com/paper/2506.06904/) (2025)
- [Three factor delay learning rules for spiking neural networks](https://mlsanity.com/paper/2601.00668/) (2026)
- [Neko: a Library for Exploring Neuromorphic Learning Rules](https://mlsanity.com/paper/2105.00324/) (2021)
- [BP(λ): Online Learning via Synthetic Gradients](https://mlsanity.com/paper/2401.07044/) (2024)

## Abstract (arXiv)

We present a framework for compactly summarizing many recent results in efficient and/or biologically plausible online training of recurrent neural networks (RNN). The framework organizes algorithms according to several criteria: (a) past vs. future facing, (b) tensor structure, (c) stochastic vs. deterministic, and (d) closed form vs. numerical. These axes reveal latent conceptual connections among several recent advances in online learning. Furthermore, we provide novel mathematical intuitions for their degree of success. Testing various algorithms on two synthetic tasks shows that performances cluster according to our criteria. Although a similar clustering is also observed for gradient alignment, alignment with exact methods does not alone explain ultimate performance, especially for stochastic algorithms. This suggests the need for better comparison metrics.