# Capacity of the Hebbian-Hopfield network associative memory

arXiv: https://arxiv.org/abs/2403.01907 · HTML: https://mlsanity.com/paper/2403.01907/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 32 references that are in the map.
- First posted on arXiv 4 Mar 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Another look at the Gardner problem](https://mlsanity.com/paper/1306.3979/) (2013)
- [Fully lifted random duality theory](https://mlsanity.com/paper/2312.00070/) (2023)
- [Discrete perceptrons](https://mlsanity.com/paper/1306.4375/) (2013)
- [Lifting ℓ1-optimization strong and sectional thresholds](https://mlsanity.com/paper/1306.3770/) (2013)
- [Lifting/lowering Hopfield models ground state energies](https://mlsanity.com/paper/1306.3975/) (2013)
- [Binary perceptrons capacity via fully lifted random duality theory](https://mlsanity.com/paper/2312.00073/) (2023)
- [Spherical perceptron as a storage memory with limited errors](https://mlsanity.com/paper/1306.3809/) (2013)
- [Fl RDT based ultimate lowering of the negative spherical perceptron capacity](https://mlsanity.com/paper/2312.16531/) (2023)
- [Dense Hebbian neural networks: a replica symmetric picture of supervised learning](https://mlsanity.com/paper/2212.00606/) (2022)
- [Exact capacity of the wide hidden layer treelike neural networks with generic activations](https://mlsanity.com/paper/2402.05719/) (2024)

## Built on by (most cited first, 1 of 1)

- [CLuP practically achieves ~ 1.77 positive and ~ 0.33 negative Hopfield model ground state free energy](https://mlsanity.com/paper/2507.22396/) (2025)

## Abstract (arXiv)

In citeHop82, Hopfield introduced a Hebbian learning rule based neural network model and suggested how it can efficiently operate as an associative memory. Studying random binary patterns, he also uncovered that, if a small fraction of errors is tolerated in the stored patterns retrieval, the capacity of the network (maximal number of memorized patterns, m) scales linearly with each pattern's size, n. Moreover, he famously predicted αc=limnrightarrow∞fracmn≈ 0.14. We study this very same scenario with two famous pattern's basins of attraction: textbf(i) The AGS one from citeAmiGutSom85; and textbf(ii) The NLT one from citeNewman88,Louk94,Louk94a,Louk97,Tal98. Relying on the fully lifted random duality theory (fl RDT) from citeStojnicflrdt23, we obtain the following explicit capacity characterizations on the first level of lifting: beginequation αc(AGS,1) = left ( max_δ∈ left ( 0,frac12right ) frac1-2δ√2 mboxerfinv left ( 1-2δright ) - frac2√2π e^-left ( mboxerfinvleft ( 1-2δ right )right )2right )2 ≈ 0.137906 endequation beginequation αc(NLT,1) = fracmboxerf(x)22x2-1+mboxerf(x)2 ≈ 0.129490, quad 1-mboxerf(x)2- frac2mboxerf(x)e-x2√πx+frac2e-2x2π=0. endequation A substantial numerical work gives on the second level of lifting αc(AGS,2) ≈ 0.138186 and αc(NLT,2) ≈ 0.12979, effectively uncovering a remarkably fast lifting convergence. Moreover, the obtained AGS characterizations exactly match the replica symmetry based ones of citeAmiGutSom85 and the corresponding symmetry breaking ones of citeSteKuh94.