mlsanityData to 14 Oct 2026

Phases in a class of associative memories via hidden neurons

arXiv 2609.10976 ↗First posted 10 Sep 2026cs.LG cond-mat.dis-nn cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

Associative memory in the Hopfield network is attractor dynamics in a disordered many-body system, and higher-order and exponential extensions turn its retrieval update into softmax attention. The polynomial and exponential regimes have been analyzed by different methods, with no common architecture in which to ask what fixes the storage scale. In this paper we study the bipartite architecture of Krotov and Hopfield, which we call the class H, whose model is fixed by a Lagrangian for each layer, taking the hidden neurons as the order parameter of retrieval. At polynomial load the replica method yields the replica-symmetric phase diagrams and closed-form capacities, and the crosstalk moment is common to Ising and spherical visible neurons, so their differences come from the visible entropy. With a softmax hidden layer the load is exponential, and a copy representation maps the thermodynamics onto random-energy-model counting, with paramagnetic, condensed, and frozen phases. Heating destabilizes retrieval by quantized reassignments of attention, and typical Gaussian patterns remain metastable at every load. The regimes differ in their crosstalk statistics, central-limit at polynomial load and large-deviation at exponential load, and the class H splits retrieval into two roles, the visible Lagrangian fixing stability and the hidden one the storage scale, two axes that may also guide the design of new Lagrangians.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 59 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

14 of 59 references.

References · 14

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Hopfield Networks is All You Needcited by 475
  3. 2016Dense Associative Memory for Pattern Recognitioncited by 236
  4. 2020Large Associative Memory Problem in Neurobiology and Machine Learningcited by 112
  5. 2023Energy Transformercited by 71
  6. 2021Hierarchical Associative Memorycited by 34
  7. 2017Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priorscited by 25
  8. 2024Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codescited by 19
  9. 2022Attention in a family of Boltzmann machines emerging from modern Hopfield networkscited by 7
  10. 2026A Biologically Plausible Dense Associative Memory with Exponential Capacitycited by 6
  11. 2024Hierarchical Associative Memory, Parallelized MLP-Mixer, and Symmetry Breakingcited by 5
  12. 2018Replica Symmetry Breaking in Bipartite Spin Glasses and Neural Networkscited by 3
2 more
  1. 2023Exploring the Temperature-Dependent Phase Transition in Modern Hopfield Networkscited by 1
  2. 2026Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memorycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Hierarchical Prototype Emergence in Modern Hopfield Models
  2. 2026High-Capacity Generalized Hopfield Networks
  3. 2025Memorization to Generalization: Emergence of Diffusion Models from Associative Memory
  4. 2026Stochastic Attention via Langevin Dynamics on the Modern Hopfield Energy
  5. 2024Dense Hopfield Networks in the Teacher-Student Setting
  6. 2023In search of dispersed memories: Generative diffusion models are associative memory networks
  7. 2026Finite-size scaling of hetero-associative retrieval in continuous-signal-driven Ising spin systems
  8. 2026Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data