mlsanityData to 14 Oct 2026

Random Features Hopfield Networks generalize retrieval to previously unseen examples

arXiv 2407.05658 ↗First posted 8 Jul 2024cond-mat.dis-nn cs.LG cs.NEcited by 8

Not on the map. Why?

Abstract (arXiv)

It has been recently shown that a learning transition happens when a Hopfield Network stores examples generated as superpositions of random features, where new attractors corresponding to such features appear in the model. In this work we reveal that the network also develops attractors corresponding to previously unseen examples generated with the same set of features. We explain this surprising behaviour in terms of spurious states of the learned features: we argue that, increasing the number of stored examples beyond the learning transition, the model also learns to mix the features to represent both stored and previously unseen examples. We support this claim with the computation of the phase diagram of the model.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 8 Jul 2024.

Builds on

18 of 35 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Layer Normalizationcited by 5,669
  3. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 657
  4. 2020Hopfield Networks is All You Needcited by 475
  5. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 413
  6. 2016Dense Associative Memory for Pattern Recognitioncited by 236
  7. 2024Dynamical Regimes of Diffusion Modelscited by 142
  8. 2020Generalisation error in learning with random features and the hidden manifold modelcited by 127
  9. 2020The Gaussian equivalence of generative models for learning with shallow neural networkscited by 113
  10. 2020Large Associative Memory Problem in Neurobiology and Machine Learningcited by 112
  11. 2023In search of dispersed memories: Generative diffusion models are associative memory networkscited by 44
  12. 2021Learning through atypical "phase transitions" in overparameterized neural networkscited by 21
6 more
  1. 2021The emergence of a concept in shallow neural networkscited by 20
  2. 2018Dreaming neural networks: forgetting spurious memories and reinforcing pure onescited by 15
  3. 2023Storage and Learning phase transitions in the Random-Features Hopfield Modelcited by 11
  4. 2022Dense Hebbian neural networks: a replica symmetric picture of supervised learningcited by 10
  5. 2023Regularization, early-stopping and dreaming: a Hopfield-like setup to address generalization and overfittingcited by 3
  6. 2023Parallel Learning by Multitasking Neural Networkscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 3.

8 papers.

  1. 2025Memorization to Generalization: Emergence of Diffusion Models from Associative Memorycited by 36
  2. 2024Dense Associative Memory Through the Lens of Random Featurescited by 8
  3. 2024Self-attention as an attractor network: transient memories without backpropagationcited by 3
  4. 2025Pseudo-likelihood produces associative memories able to generalize, even for asymmetric couplingscited by 2
  5. 2026Memorisation, convergence and generalisation in generative modelscited by 1
  6. 2025Implicit Bias and Invariance: How Hopfield Networks Efficiently Learn Graph Orbitscited by 0
  7. 2026Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Datacited by 0
  8. 2026Exponential Capacity in Multilayer Hetero-Associative Neural Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Dense Hopfield Networks in the Teacher-Student Setting
  2. 2026Feature Lottery? A Bifurcation Theory of Concept Emergence
  3. 2026Phases in a class of associative memories via hidden neurons
  4. 2023Hopfield model with planted patterns: a teacher-student self-supervised learning model
  5. 2026Continual Learning in Modern Hopfield Networks with an Application to Diffusion Models
  6. 2026Emergent phases of superposition: from partial to full representation
  7. 2026Hierarchical Prototype Emergence in Modern Hopfield Models
  8. 2023On information captured by neural networks: connections with memorization and generalization