# Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex

arXiv: https://arxiv.org/abs/1511.00083 · HTML: https://mlsanity.com/paper/1511.00083/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2015.

## Built on by (most cited first, 30 of 45)

- [How Linguistics Learned to Stop Worrying and Love the Language Models](https://mlsanity.com/paper/2501.17047/) (2025)
- [Attention Approximates Sparse Distributed Memory](https://mlsanity.com/paper/2111.05498/) (2021)
- [Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments](https://mlsanity.com/paper/2201.00042/) (2021)
- [Long Sequence Hopfield Memory](https://mlsanity.com/paper/2306.04532/) (2023)
- [Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks](https://mlsanity.com/paper/2112.13896/) (2021)
- [EvoBrain: Dynamic Multi-Channel EEG Graph Modeling for Time-Evolving Brain Networks](https://mlsanity.com/paper/2509.15857/) (2025)
- [Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computing](https://mlsanity.com/paper/2206.13336/) (2022)
- [Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks](https://mlsanity.com/paper/2301.09245/) (2023)
- [One Neuron Saved Is One Neuron Earned: On Parametric Efficiency of Quadratic Networks](https://mlsanity.com/paper/2303.06316/) (2023)
- [TS-LIF: A Temporal Segment Spiking Neuron Network for Time Series Forecasting](https://mlsanity.com/paper/2503.05108/) (2025)
- [Grid Cell Path Integration For Movement-Based Visual Object Recognition](https://mlsanity.com/paper/2102.09076/) (2021)
- [The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks](https://mlsanity.com/paper/2306.16922/) (2023)
- [Neuroscience-Inspired Algorithms for the Predictive Maintenance of Manufacturing Systems](https://mlsanity.com/paper/2102.11450/) (2021)
- [An active dendritic tree can mitigate fan-in limitations in superconducting neurons](https://mlsanity.com/paper/2107.05777/) (2021)
- [The Thousand Brains Theory 2.0: An Extension for the Long-Range Connections of the Neocortical Heterarchy](https://mlsanity.com/paper/2507.05888/) (2025)
- [SPARTA: Advancing Sparse Attention in Spiking Neural Networks via Spike-Timing-Based Prioritization](https://mlsanity.com/paper/2508.01646/) (2025)
- [A Macrocolumn Architecture Implemented with Spiking Neurons](https://mlsanity.com/paper/2207.05081/) (2022)
- [The Construction of Reality in an AI: A Review](https://mlsanity.com/paper/2302.05448/) (2023)
- [Learning Successor Features with Distributed Hebbian Temporal Memory](https://mlsanity.com/paper/2310.13391/) (2023)
- [One Framework to Rule Them All: Unifying Multimodal Tasks with LLM Neural-Tuning](https://mlsanity.com/paper/2408.03001/) (2024)
- [Associative Knowledge Graphs for Efficient Sequence Storage and Retrieval](https://mlsanity.com/paper/2411.14480/) (2024)
- [Threshold Adaptation in Spiking Networks Enables Shortest Path Finding and Place Disambiguation](https://mlsanity.com/paper/2503.21795/) (2025)
- [On the Parallels Between Evolutionary Theory and the State of AI](https://mlsanity.com/paper/2505.23774/) (2025)
- [Toward Human-Level Artificial Intelligence](https://mlsanity.com/paper/2108.03793/) (2021)
- [Cloud Failure Prediction with Hierarchical Temporal Memory: An Empirical Assessment](https://mlsanity.com/paper/2110.03431/) (2021)
- [Application of Hierarchical Temporal Memory Theory for Document Categorization](https://mlsanity.com/paper/2112.14820/) (2021)
- [MARTI-4: new model of human brain, considering neocortex and basal ganglia -- learns to play Atari game by reinforcement learning on a single CPU](https://mlsanity.com/paper/2209.02387/) (2022)
- [A Brain-Inspired Sequence Learning Model based on a Logic](https://mlsanity.com/paper/2308.12486/) (2023)
- [Large-scale Foundation Models and Generative AI for BigData Neuroscience](https://mlsanity.com/paper/2310.18377/) (2023)
- [Implementing Online Reinforcement Learning with Clustering Neural Networks](https://mlsanity.com/paper/2402.18472/) (2024)

## Abstract (arXiv)

Neocortical neurons have thousands of excitatory synapses. It is a mystery how neurons integrate the input from so many synapses and what kind of large-scale network behavior this enables. It has been previously proposed that non-linear properties of dendrites enable neurons to recognize multiple patterns. In this paper we extend this idea by showing that a neuron with several thousand synapses arranged along active dendrites can learn to accurately and robustly recognize hundreds of unique patterns of cellular activity, even in the presence of large amounts of noise and pattern variation. We then propose a neuron model where some of the patterns recognized by a neuron lead to action potentials and define the classic receptive field of the neuron, whereas the majority of the patterns recognized by a neuron act as predictions by slightly depolarizing the neuron without immediately generating an action potential. We then present a network model based on neurons with these properties and show that the network learns a robust model of time-based sequences. Given the similarity of excitatory neurons throughout the neocortex and the importance of sequence memory in inference and behavior, we propose that this form of sequence memory is a universal property of neocortical tissue. We further propose that cellular layers in the neocortex implement variations of the same sequence memory algorithm to achieve different aspects of inference and behavior. The neuron and network models we introduce are robust over a wide range of parameters as long as the network uses a sparse distributed code of cellular activations. The sequence capacity of the network scales linearly with the number of synapses on each neuron. Thus neurons need thousands of synapses to learn the many temporal patterns in sensory stimuli and motor sequences.