# Sufficient conditions for offline reactivation in recurrent neural networks

arXiv: https://arxiv.org/abs/2505.17003 · HTML: https://mlsanity.com/paper/2505.17003/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 62 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 7 of 7)

- [Memory Efficient Experience Replay for Streaming Learning](https://mlsanity.com/paper/1809.05922/) (2018)
- [Attractor and integrator networks in the brain](https://mlsanity.com/paper/2112.03978/) (2021)
- [Emergence of grid-like representations by training recurrent neural networks to perform spatial localization](https://mlsanity.com/paper/1803.07770/) (2018)
- [Online Continual Learning for Embedded Devices](https://mlsanity.com/paper/2203.10681/) (2022)
- [Gated recurrent units viewed through the lens of continuous time dynamical systems](https://mlsanity.com/paper/1906.01005/) (2019)
- [Learning cortical representations through perturbed and adversarial dreaming](https://mlsanity.com/paper/2109.04261/) (2021)
- [Emergence of functional and structural properties of the head direction system by optimization of recurrent neural networks](https://mlsanity.com/paper/1912.10189/) (2019)

## Built on by (most cited first, 2 of 2)

- [Paradoxical noise preference in RNNs](https://mlsanity.com/paper/2601.04539/) (2026)
- [Leakage and Second-Order Dynamics Improve Hippocampal RNN Replay](https://mlsanity.com/paper/2602.18401/) (2026)

## Abstract (arXiv)

During periods of quiescence, such as sleep, neural activity in many brain circuits resembles that observed during periods of task engagement. However, the precise conditions under which task-optimized networks can autonomously reactivate the same network states responsible for online behavior is poorly understood. In this study, we develop a mathematical framework that outlines sufficient conditions for the emergence of neural reactivation in circuits that encode features of smoothly varying stimuli. We demonstrate mathematically that noisy recurrent networks optimized to track environmental state variables using change-based sensory information naturally develop denoising dynamics, which, in the absence of input, cause the network to revisit state configurations observed during periods of online activity. We validate our findings using numerical experiments on two canonical neuroscience tasks: spatial position estimation based on self-motion cues, and head direction estimation based on angular velocity cues. Overall, our work provides theoretical support for modeling offline reactivation as an emergent consequence of task optimization in noisy neural circuits.