# Slow manifolds in recurrent networks encode working memory efficiently and robustly

arXiv: https://arxiv.org/abs/2101.03163 · HTML: https://mlsanity.com/paper/2101.03163/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 51 references that are in the map.
- First posted on arXiv 8 Jan 2021.

## Builds on (most cited first, 8 of 8)

- [Universality and individuality in neural dynamics across large populations of recurrent networks](https://mlsanity.com/paper/1907.08549/) (2019)
- [Linking connectivity, dynamics and computations in low-rank recurrent neural networks](https://mlsanity.com/paper/1711.09672/) (2017)
- [Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics](https://mlsanity.com/paper/1906.10720/) (2019)
- [Artificial neural networks for neuroscientists: A primer](https://mlsanity.com/paper/2006.01001/) (2020)
- [Recurrent Network Models Of Sequence Generation And Memory](https://mlsanity.com/paper/1603.04687/) (2016)
- [Dynamics of random recurrent networks with correlated low-rank structure](https://mlsanity.com/paper/1909.04358/) (2019)
- [Coding with transient trajectories in recurrent neural networks](https://mlsanity.com/paper/1811.07592/) (2018)
- [One step back, two steps forward: interference and learning in recurrent neural networks](https://mlsanity.com/paper/1805.09603/) (2018)

## Built on by (most cited first, 1 of 1)

- [Divisive Normalization Shapes Low-Rank Slow Manifolds for Continuous Working Memory](https://mlsanity.com/paper/2608.01947/) (2026)

## Abstract (arXiv)

Working memory is a cognitive function involving the storage and manipulation of latent information over brief intervals of time, thus making it crucial for context-dependent computation. Here, we use a top-down modeling approach to examine network-level mechanisms of working memory, an enigmatic issue and central topic of study in neuroscience and machine intelligence. We train thousands of recurrent neural networks on a working memory task and then perform dynamical systems analysis on the ensuing optimized networks, wherein we find that four distinct dynamical mechanisms can emerge. In particular, we show the prevalence of a mechanism in which memories are encoded along slow stable manifolds in the network state space, leading to a phasic neuronal activation profile during memory periods. In contrast to mechanisms in which memories are directly encoded at stable attractors, these networks naturally forget stimuli over time. Despite this seeming functional disadvantage, they are more efficient in terms of how they leverage their attractor landscape and paradoxically, are considerably more robust to noise. Our results provide new dynamical hypotheses regarding how working memory function is encoded in both natural and artificial neural networks.