# Unbiased Online Recurrent Optimization

arXiv: https://arxiv.org/abs/1702.05043 · HTML: https://mlsanity.com/paper/1702.05043/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Feb 2017.

## Built on by (most cited first, 30 of 47)

- [Resurrecting Recurrent Neural Networks for Long Sequences](https://mlsanity.com/paper/2303.06349/) (2023)
- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [Predictive Coding: a Theoretical and Experimental Review](https://mlsanity.com/paper/2107.12979/) (2021)
- [Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies](https://mlsanity.com/paper/2112.13835/) (2021)
- [Online Training Through Time for Spiking Neural Networks](https://mlsanity.com/paper/2210.04195/) (2022)
- [Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization](https://mlsanity.com/paper/2201.12250/) (2022)
- [Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation](https://mlsanity.com/paper/2205.03076/) (2022)
- [Online learning of long-range dependencies](https://mlsanity.com/paper/2305.15947/) (2023)
- [Exploring the Promise and Limits of Real-Time Recurrent Learning](https://mlsanity.com/paper/2305.19044/) (2023)
- [Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment](https://mlsanity.com/paper/2312.09257/) (2023)
- [Scalable Real-Time Recurrent Learning Using Columnar-Constructive Networks](https://mlsanity.com/paper/2302.05326/) (2023)
- [Real-Time Recurrent Learning using Trace Units in Reinforcement Learning](https://mlsanity.com/paper/2409.01449/) (2024)
- [Amortized Proximal Optimization](https://mlsanity.com/paper/2203.00089/) (2022)
- [Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Single](https://mlsanity.com/paper/2304.11153/) (2023)
- [Learning long range dependencies through time reversal symmetry breaking](https://mlsanity.com/paper/2506.05259/) (2025)
- [Real-Time Recurrent Reinforcement Learning](https://mlsanity.com/paper/2311.04830/) (2023)
- [Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies](https://mlsanity.com/paper/2304.12180/) (2023)
- [Online hyperparameter optimization by real-time recurrent learning](https://mlsanity.com/paper/2102.07813/) (2021)
- [Prediction of the Position of External Markers Using a Recurrent Neural Network Trained With Unbiased Online Recurrent Optimization for Safe Lung Cancer Radiotherapy](https://mlsanity.com/paper/2106.01100/) (2021)
- [Scaling Recurrent Neural Networks to a Billion Parameters with Zero-Order Optimization](https://mlsanity.com/paper/2505.17852/) (2025)
- [Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules](https://mlsanity.com/paper/2206.00823/) (2022)
- [Learning Agent State Online with Recurrent Generate-and-Test](https://mlsanity.com/paper/2112.15236/) (2021)
- [Pavlovian Signalling with General Value Functions in Agent-Agent Temporal Decision Making](https://mlsanity.com/paper/2201.03709/) (2022)
- [An Empirical Analysis of Recurrent Learning Algorithms In Neural Lossy Image Compression Systems](https://mlsanity.com/paper/2201.11782/) (2022)
- [The Frost Hollow Experiments: Pavlovian Signalling as a Path to Coordination and Communication Between Agents](https://mlsanity.com/paper/2203.09498/) (2022)
- [Estimating Post-Synaptic Effects for Online Training of Feed-Forward SNNs](https://mlsanity.com/paper/2311.16151/) (2023)
- [Real-time respiratory motion forecasting with online learning of recurrent neural networks for accurate targeting in externally guided radiotherapy](https://mlsanity.com/paper/2403.01607/) (2024)
- [Bio-Inspired Mamba: Temporal Locality and Bioplausible Learning in Selective State Space Models](https://mlsanity.com/paper/2409.11263/) (2024)
- [Mind the truncation gap: challenges of learning on dynamic graphs with recurrent architectures](https://mlsanity.com/paper/2412.21046/) (2024)

## Abstract (arXiv)

The novel Unbiased Online Recurrent Optimization (UORO) algorithm allows for online learning of general recurrent computational graphs such as recurrent network models. It works in a streaming fashion and avoids backtracking through past activations and inputs. UORO is computationally as costly as Truncated Backpropagation Through Time (truncated BPTT), a widespread algorithm for online learning of recurrent networks. UORO is a modification of NoBackTrack that bypasses the need for model sparsity and makes implementation easy in current deep learning frameworks, even for complex models. Like NoBackTrack, UORO provides unbiased gradient estimates; unbiasedness is the core hypothesis in stochastic gradient descent theory, without which convergence to a local optimum is not guaranteed. On the contrary, truncated BPTT does not provide this property, leading to possible divergence. On synthetic tasks where truncated BPTT is shown to diverge, UORO converges. For instance, when a parameter has a positive short-term but negative long-term influence, truncated BPTT diverges unless the truncation span is very significantly longer than the intrinsic temporal range of the interactions, while UORO performs well thanks to the unbiasedness of its gradients.