# Learning Sequence Attractors in Recurrent Networks with Hidden Neurons

arXiv: https://arxiv.org/abs/2404.02729 · HTML: https://mlsanity.com/paper/2404.02729/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 43 references that are in the map.
- First posted on arXiv 3 Apr 2024.

## Builds on (most cited first, 7 of 7)

- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation](https://mlsanity.com/paper/1407.7906/) (2014)
- [Attractor and integrator networks in the brain](https://mlsanity.com/paper/2112.03978/) (2021)
- [Recurrent Network Models Of Sequence Generation And Memory](https://mlsanity.com/paper/1603.04687/) (2016)
- [Sequential Memory with Temporal Predictive Coding](https://mlsanity.com/paper/2305.11982/) (2023)
- [Long Sequence Hopfield Memory](https://mlsanity.com/paper/2306.04532/) (2023)

## Built on by (most cited first, 1 of 1)

- [A General Close-loop Predictive Coding Framework for Auditory Working Memory](https://mlsanity.com/paper/2503.12506/) (2025)

## Abstract (arXiv)

The brain is targeted for processing temporal sequence information. It remains largely unclear how the brain learns to store and retrieve sequence memories. Here, we study how recurrent networks of binary neurons learn sequence attractors to store predefined pattern sequences and retrieve them robustly. We show that to store arbitrary pattern sequences, it is necessary for the network to include hidden neurons even though their role in displaying sequence memories is indirect. We develop a local learning algorithm to learn sequence attractors in the networks with hidden neurons. The algorithm is proven to converge and lead to sequence attractors. We demonstrate that the network model can store and retrieve sequences robustly on synthetic and real-world datasets. We hope that this study provides new insights in understanding sequence memory and temporal information processing in the brain.