# Associative Memory for Non-Stationary Environments: A Self-Sizing Generalization of Hopfield Networks

arXiv: https://arxiv.org/abs/2507.10443 · HTML: https://mlsanity.com/paper/2507.10443/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 45 references that are in the map.
- First posted on arXiv 8 Jul 2025.

## Builds on (most cited first, 10 of 10)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Lifelong Learning with Dynamically Expandable Networks](https://mlsanity.com/paper/1708.01547/) (2017)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning](https://mlsanity.com/paper/2001.00689/) (2020)
- [On the Stability-Plasticity Dilemma of Class-Incremental Learning](https://mlsanity.com/paper/2304.01663/) (2023)

## Built on by (most cited first, 1 of 1)

- [Cycle is All You Need: More Is Different](https://mlsanity.com/paper/2509.21340/) (2025)

## Abstract (arXiv)

The Hopfield network made associative memory (AM) the model system of neural computation, but it solves the problem only for a stationary world: a fixed set of memories, stored once into frozen weights. Real environments are non-stationary (e.g., memories arrive over time, drift, recur, and must be told apart from noise), where the classical formulation fails by catastrophic interference (the palimpsest problem) and by a capacity fixed in advance. We argue that this is not a peripheral limitation but the crux: under non-stationarity, memory and learning cease to be separate problems, and adaptation, rather than one-shot optimization, becomes the operative capacity. We give a fresh formulation of the AM problem for non-stationary environments and a self-sizing continual associative memory that generalizes Hopfield's: it stores new memories without erasing old ones (no forgetting), re-binds drifting and recurring memories, allocates a genuinely new memory only for true novelty, and grows its store to the environment's intrinsic memory demand and no further. We rigorously show that this demand is the Urysohn width of the problem and can be estimated from data via a contrastive-similarity (CS) operator. The memory's size converges to this capacity online with no preset value and no validation search, matching an oracle capacity search. We use experiments with synthetic datasets to show that the generalization buys self-sizing and retention under non-stationarity, not higher per-item recall fidelity, on which it matches strong baselines.