mlsanityData to 14 Oct 2026

Dense Associative Memory with Epanechnikov Energy

arXiv 2506.10801 ↗First posted 12 Jun 2025cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR), inspired by optimal kernel density estimation. Unlike the common log-sum-exponential (LSE) function, LSR is based on the Epanechnikov kernel and enables exact memory retrieval with exponential capacity without requiring exponential separation functions. Moreover, it introduces abundant additional emergent local minima while preserving perfect pattern recovery -- a characteristic previously unseen in DenseAM literature. Empirical results show that LSR energy has significantly more local minima (memories) that have comparable log-likelihood to LSE-based models. Analysis of LSR's emergent memories on image datasets reveals a degree of creativity and novelty, hinting at this method's potential for both large-scale memory storage and generative tasks.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 16 of its 35 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

16 of 35 references.

References · 16

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Hopfield Networks is All You Needcited by 475
  4. 2021Deep learning: a statistical viewpointcited by 240
  5. 2016Dense Associative Memory for Pattern Recognitioncited by 236
  6. 2021Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationcited by 132
  7. 2020Large Associative Memory Problem in Neurobiology and Machine Learningcited by 112
  8. 2018Does data interpolation contradict statistical optimality?cited by 95
  9. 2017Dense Associative Memory is Robust to Adversarial Inputscited by 45
  10. 2023On Sparse Modern Hopfield Modelcited by 42
  11. 2024Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Modelscited by 38
  12. 2021Hierarchical Associative Memorycited by 34
4 more
  1. 2017Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priorscited by 25
  2. 2024Sparse and Structured Hopfield Networkscited by 13
  3. 2024Dense Associative Memory Through the Lens of Random Featurescited by 8
  4. 2024On the number of modes of Gaussian kernel density estimatorscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 9.

11 papers.

  1. 2025Dense associative memory for Gaussian distributionscited by 2
  2. 2025Memory-Integrated Reconfigurable Adapters: A Unified Framework for Settings with Multiple Taskscited by 2
  3. 2026Sparse Attention as Compact Kernel Regressioncited by 2
  4. 2026Energy-Based Dynamical Models for Neurocomputation, Learning, and Optimizationcited by 1
  5. 2026Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memorycited by 1
  6. 2026Graph Hopfield Networks: Energy-Based Node Classification with Associative Memorycited by 0
  7. 2026Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernelscited by 0
  8. 2026Balancing Fidelity and Diversity in Diffusion Models via Symmetric Attention Decomposition: Hopfield Perspectivecited by 0
  9. 2026Hierarchical Prototype Emergence in Modern Hopfield Modelscited by 0
  10. 2026Graph Memory: Spectral Associative Memory via Dirichlet Energycited by 0
  11. 2026Intrinsic Associative Memory on Riemannian Manifolds: Curvature, Capacity, and Emergent Modescited by 0