# Recurrent Network Models Of Sequence Generation And Memory

arXiv: https://arxiv.org/abs/1603.04687 · HTML: https://mlsanity.com/paper/1603.04687/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2016.

## Built on by (most cited first, 21 of 21)

- [The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware](https://mlsanity.com/paper/2106.07030/) (2021)
- [Long Sequence Hopfield Memory](https://mlsanity.com/paper/2306.04532/) (2023)
- [DeepSITH: Efficient Learning via Decomposition of What and When Across Time Scales](https://mlsanity.com/paper/2104.04646/) (2021)
- [Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience](https://mlsanity.com/paper/2408.12664/) (2024)
- [Uncovering the Computational Roles of Nonlinearity in Sequence Modeling Using Almost-Linear RNNs](https://mlsanity.com/paper/2506.07919/) (2025)
- [Efficient and robust multi-task learning in the brain with modular latent primitives](https://mlsanity.com/paper/2105.14108/) (2021)
- [Spectrum of non-Hermitian deep-Hebbian neural networks](https://mlsanity.com/paper/2208.11411/) (2022)
- [Episodic memory governs choices: An RNN-based reinforcement learning model for decision-making task](https://mlsanity.com/paper/2103.03679/) (2021)
- [Deep Continuous Networks](https://mlsanity.com/paper/2402.01557/) (2024)
- [Slow manifolds in recurrent networks encode working memory efficiently and robustly](https://mlsanity.com/paper/2101.03163/) (2021)
- [Learning Sequence Attractors in Recurrent Networks with Hidden Neurons](https://mlsanity.com/paper/2404.02729/) (2024)
- [A ghost mechanism: An analytical model of abrupt learning in recurrent networks](https://mlsanity.com/paper/2501.02378/) (2025)
- [Bubblewrap: Online tiling and real-time flow prediction on neural manifolds](https://mlsanity.com/paper/2108.13941/) (2021)
- [A Versatile Hub Model For Efficient Information Propagation And Feature Selection](https://mlsanity.com/paper/2307.02398/) (2023)
- [Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks](https://mlsanity.com/paper/2410.03972/) (2024)
- [Dissecting Larval Zebrafish Hunting using Deep Reinforcement Learning Trained RNN Agents](https://mlsanity.com/paper/2510.03699/) (2025)
- [Hierarchical temporal receptive windows and zero-shot timescale generalization in biologically constrained scale-invariant deep networks](https://mlsanity.com/paper/2601.02618/) (2026)
- [Systems Explaining Systems: A Framework for Intelligence and Consciousness](https://mlsanity.com/paper/2601.04269/) (2026)
- [Drift-Diffusion Matching: Embedding dynamics in latent manifolds of asymmetric neural networks](https://mlsanity.com/paper/2602.14885/) (2026)
- [Identifying Connectivity Distributions from Neural Dynamics Using Flows](https://mlsanity.com/paper/2603.26506/) (2026)
- [When and Where: A Model Hippocampal Network Unifies Formation of Time Cells and Place Cells](https://mlsanity.com/paper/2604.00036/) (2026)

## Abstract (arXiv)

Sequential activation of neurons is a common feature of network activity during a variety of behaviors, including working memory and decision making. Previous network models for sequences and memory emphasized specialized architectures in which a principled mechanism is pre-wired into their connectivity. Here we demonstrate that, starting from random connectivity and modifying a small fraction of connections, a largely disordered recur- rent network can produce sequences and implement working memory efficiently. We use this process, called Partial In-Network Training (PINning), to model and match cellular resolution imaging data from the posterior parietal cortex during a virtual memory- guided two-alternative forced-choice task. Analysis of the connectivity reveals that sequences propagate by the cooperation between recurrent synaptic interactions and external inputs, rather than through feedforward or asymmetric connections. Together our results suggest that neural sequences may emerge through learning from largely unstructured network architectures.