# Dense Hebbian neural networks: a replica symmetric picture of unsupervised learning

arXiv: https://arxiv.org/abs/2211.14067 · HTML: https://mlsanity.com/paper/2211.14067/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 48 references that are in the map.
- First posted on arXiv 25 Nov 2022.

## Builds on (most cited first, 6 of 6)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Dense Associative Memory is Robust to Adversarial Inputs](https://mlsanity.com/paper/1701.00939/) (2017)
- [The emergence of a concept in shallow neural networks](https://mlsanity.com/paper/2109.00454/) (2021)
- [Supervised Hebbian Learning](https://mlsanity.com/paper/2203.01304/) (2022)
- [Neural networks with redundant representation: detecting the undetectable](https://mlsanity.com/paper/1911.12689/) (2019)

## Built on by (most cited first, 10 of 10)

- [Long Sequence Hopfield Memory](https://mlsanity.com/paper/2306.04532/) (2023)
- [Parallel Learning by Multitasking Neural Networks](https://mlsanity.com/paper/2308.04106/) (2023)
- [Hebbian Learning from First Principles](https://mlsanity.com/paper/2401.07110/) (2024)
- [Explosive neural networks via higher-order interactions in curved statistical manifolds](https://mlsanity.com/paper/2408.02326/) (2024)
- [The effect of priors on Learning with Restricted Boltzmann Machines](https://mlsanity.com/paper/2412.02623/) (2024)
- [Statistical Mechanics of Learning via Reverberation in Bidirectional Associative Memories](https://mlsanity.com/paper/2307.08365/) (2023)
- [Neural Learning Rules from Associative Networks Theory](https://mlsanity.com/paper/2503.19922/) (2025)
- [Unsupervised and Supervised learning by Dense Associative Memory under replica symmetry breaking](https://mlsanity.com/paper/2312.09638/) (2023)
- [Beyond Disorder: Unveiling Cooperativeness in Multidirectional Associative Memories](https://mlsanity.com/paper/2503.04454/) (2025)
- [Temporal Complexity and Self-Organization in an Exponential Dense Associative Memory Model](https://mlsanity.com/paper/2601.11478/) (2026)

## Abstract (arXiv)

We consider dense, associative neural-networks trained with no supervision and we investigate their computational capabilities analytically, via a statistical-mechanics approach, and numerically, via Monte Carlo simulations. In particular, we obtain a phase diagram summarizing their performance as a function of the control parameters such as the quality and quantity of the training dataset and the network storage, valid in the limit of large network size and structureless datasets. Moreover, we establish a bridge between macroscopic observables standardly used in statistical mechanics and loss functions typically used in the machine learning. As technical remarks, from the analytic side, we implement large deviations and stability analysis within Guerra's interpolation to tackle the not-Gaussian distributions involved in the post-synaptic potentials while, from the computational counterpart, we insert Plefka approximation in the Monte Carlo scheme, to speed up the evaluation of the synaptic tensors, overall obtaining a novel and broad approach to investigate neural networks in general.