Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory
Not on the map. Why?
Abstract (arXiv)
We study the thermodynamic memory capacity of modern Hopfield networks (Dense Associative Memory models) with continuous states under geometric constraints, extending classical analyses of pairwise associative memory. We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity M = eα N, comparing Gaussian (LSE) and Epanechnikov (LSR) kernels. For continuous neurons on an N-sphere, the geometric entropy depends solely on the spherical geometry, not the kernel. In the sharp-kernel regime, the maximum theoretical capacity α = 0.5 is achieved at zero temperature; below this threshold, a critical line separates retrieval from non-retrieval. The two kernels differ qualitatively in their phase boundary structure: for LSE, a critical line exists at all loads α > 0. For LSR, the finite support introduces a threshold αth below which no spurious patterns contribute to the noise floor, and no critical line exists -- retrieval is perfect at any temperature. These results advance the theory of high-capacity associative memory and clarify fundamental limits of retrieval robustness in modern attention-like memory architectures.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 3 of its 15 references that are in the map. First posted on arXiv 8 Apr 2026.
Builds on
3 of 15 references.
References · 3
- 2016Dense Associative Memory for Pattern Recognitioncited by 230
- 2020Large Associative Memory Problem in Neurobiology and Machine Learningcited by 110
- 2025Dense Associative Memory with Epanechnikov Energycited by 10
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Phases in a class of associative memories via hidden neurons10 Sep 2026