# Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffold

arXiv: https://arxiv.org/abs/2202.00159 · HTML: https://mlsanity.com/paper/2202.00159/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 49 references that are in the map.
- First posted on arXiv 1 Feb 2022.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Large Associative Memory Problem in Neurobiology and Machine Learning](https://mlsanity.com/paper/2008.06996/) (2020)
- [Overparameterized Neural Networks Implement Associative Memory](https://mlsanity.com/paper/1909.12362/) (2019)
- [MEMO: A Deep Network for Flexible Combination of Episodic Memories](https://mlsanity.com/paper/2001.10913/) (2020)
- [Biological learning in key-value memory networks](https://mlsanity.com/paper/2110.13976/) (2021)
- [Neural Stored-program Memory](https://mlsanity.com/paper/1906.08862/) (2019)

## Built on by (most cited first, 11 of 11)

- [Key-value memory in the brain](https://mlsanity.com/paper/2501.02950/) (2025)
- [Simplicial Hopfield networks](https://mlsanity.com/paper/2305.05179/) (2023)
- [BayesPCN: A Continually Learnable Predictive Coding Associative Memory](https://mlsanity.com/paper/2205.09930/) (2022)
- [Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models](https://mlsanity.com/paper/2309.16750/) (2023)
- [Bridging Associative Memory and Probabilistic Modeling](https://mlsanity.com/paper/2402.10202/) (2024)
- [Associative memory inspires improvements for in-context learning using a novel attention residual stream architecture](https://mlsanity.com/paper/2412.15113/) (2024)
- [Semantically-correlated memories in a dense associative model](https://mlsanity.com/paper/2404.07123/) (2024)
- [Are Grid Cells Hexagonal for Performance or by Convenience?](https://mlsanity.com/paper/2410.11886/) (2024)
- [Storing overlapping associative memories on latent manifolds in low-rank spiking networks](https://mlsanity.com/paper/2411.17485/) (2024)
- [Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamics](https://mlsanity.com/paper/2603.20656/) (2026)
- [Controlled Dynamics Attractor Transformer](https://mlsanity.com/paper/2606.15207/) (2026)

## Abstract (arXiv)

Content-addressable memory (CAM) networks, so-called because stored items can be recalled by partial or corrupted versions of the items, exhibit near-perfect recall of a small number of information-dense patterns below capacity and a 'memory cliff' beyond, such that inserting a single additional pattern results in catastrophic loss of all stored patterns. We propose a novel CAM architecture, Memory Scaffold with Heteroassociation (MESH), that factorizes the problems of internal attractor dynamics and association with external content to generate a CAM continuum without a memory cliff: Small numbers of patterns are stored with complete information recovery matching standard CAMs, while inserting more patterns still results in partial recall of every pattern, with a graceful trade-off between pattern number and pattern richness. Motivated by the architecture of the Entorhinal-Hippocampal memory circuit in the brain, MESH is a tripartite architecture with pairwise interactions that uses a predetermined set of internally stabilized states together with heteroassociation between the internal states and arbitrary external patterns. We show analytically and experimentally that for any number of stored patterns, MESH nearly saturates the total information bound (given by the number of synapses) for CAM networks, outperforming all existing CAM models.