mlsanityData to 14 Oct 2026

Dense Associative Memory for Pattern Recognition

arXiv 1606.01164 ↗First posted 3 Jun 2016cs.NE cond-mat.dis-nn cs.LGcited by 236

Not on the map. Why?

Abstract (arXiv)

A model of associative memory is studied, which stores and reliably retrieves many more patterns than the number of neurons in the network. We propose a simple duality between this dense associative memory and neural networks commonly used in deep learning. On the associative memory side of this duality, a family of models that smoothly interpolates between two limiting cases can be constructed. One limit is referred to as the feature-matching mode of pattern recognition, and the other one as the prototype regime. On the deep learning side of the duality, this family corresponds to feedforward neural networks with one hidden layer and various activation functions, which transmit the activities of the visible neurons to the hidden layer. This family of activation functions includes logistics, rectified linear units, and rectified polynomials of higher degrees. The proposed duality makes it possible to apply energy-based intuition from associative memory to analyze computational properties of neural networks with unusual activation functions - the higher rectified polynomials which until now have not been used in deep learning. The utility of the dense memories is illustrated for two test cases: the logical gate XOR and the recognition of handwritten digits from the MNIST data set.

Key facts. Cited by 236 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 21; 2023: 37; 2024: 53; 2025: 49; 2026: 60.

236 papers.

Sort:

By area:

  1. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  2. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 164
  3. 2023Birth of a Transformer: A Memory Viewpointcited by 161
  4. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  5. 2021On the Adversarial Robustness of Vision Transformerscited by 106
  6. 2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 89
  7. 2025Test-time regression: a unifying framework for designing sequence models with associative memorycited by 74
  8. 2023Energy Transformercited by 71
  9. 2025ATLAS: Learning to Optimally Memorize the Context at Test Timecited by 66
  10. 2025It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimizationcited by 64
  11. 2021Relating transformers to models and neural representations of the hippocampal formationcited by 62
  12. 2022Hebbian Deep Learning Without Feedbackcited by 61
38 more
  1. 2022Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Modelscited by 55
  2. 2021Associative Memories via Predictive Codingcited by 44
  3. 2023In search of dispersed memories: Generative diffusion models are associative memory networkscited by 44
  4. 2023Conformal Prediction for Time Series with Modern Hopfield Networkscited by 42
  5. 2023On Sparse Modern Hopfield Modelcited by 42
  6. 2022History Compression via Language Models in Reinforcement Learningcited by 39
  7. 2023The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instabilitycited by 39
  8. 2024Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Modelscited by 38
  9. 2024Outlier-Efficient Hopfield Layers for Large Transformer-Based Modelscited by 38
  10. 2023Scaling Laws for Associative Memoriescited by 35
  11. 2023STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Predictioncited by 35
  12. 2021Hierarchical Associative Memorycited by 34
  13. 2022Fine-tuning Image Transformers using Learnable Memorycited by 34
  14. 2022Learning on Arbitrary Graph Topologies via Predictive Codingcited by 33
  15. 2021SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networkscited by 30
  16. 2022Txt2Img-MHN: Remote Sensing Image Generation from Text Using Modern Hopfield Networkscited by 30
  17. 2021Attention Approximates Sparse Distributed Memorycited by 29
  18. 2024Losing dimensions: Geometric memorization in generative diffusioncited by 29
  19. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 26
  20. 2024CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memorycited by 24
  21. 2025Key-value memory in the braincited by 24
  22. 2024BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Modelcited by 21
  23. 2022Abstraction for Deep Reinforcement Learningcited by 19
  24. 2024Conv-CoA: Improving Open-domain Question Answering in Large Language Models via Conversational Chain-of-Actioncited by 19
  25. 2024Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codescited by 19
  26. 2022Torchhd: An Open Source Python Library to Support Research on Hyperdimensional Computing and Vector Symbolic Architecturescited by 17
  27. 2022A Theoretical Framework for Inference Learningcited by 17
  28. 2023Simplicial Hopfield networkscited by 16
  29. 2023Sequential Memory with Temporal Predictive Codingcited by 16
  30. 2023Long Sequence Hopfield Memorycited by 16
  31. 2024Nonparametric Modern Hopfield Modelscited by 16
  32. 2023End-to-end Differentiable Clustering with Associative Memoriescited by 15
  33. 2026Memory Caching: RNNs with Growing Memorycited by 15
  34. 2022ChiTransformer:Towards Reliable Stereo from Cuescited by 14
  35. 2022BayesPCN: A Continually Learnable Predictive Coding Associative Memorycited by 14
  36. 2021Trusted Artificial Intelligence: Towards Certification of Machine Learning Applicationscited by 13
  37. 2023Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Modelscited by 13
  38. 2023In-Context Exemplars as Clues to Retrieving from Large Associative Memorycited by 13

50 most cited of 236 All 236 (JSON)