# Dense Associative Memory is Robust to Adversarial Inputs

arXiv: https://arxiv.org/abs/1701.00939 · HTML: https://mlsanity.com/paper/1701.00939/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jan 2017.

## Built on by (most cited first, 30 of 45)

- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [On the Adversarial Robustness of Vision Transformers](https://mlsanity.com/paper/2103.15670/) (2021)
- [Hierarchical Associative Memory](https://mlsanity.com/paper/2107.06446/) (2021)
- [Fine-tuning Image Transformers using Learnable Memory](https://mlsanity.com/paper/2203.15243/) (2022)
- [Energy-Based Cross Attention for Bayesian Context Update in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2306.09869/) (2023)
- [Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives](https://mlsanity.com/paper/2307.16851/) (2023)
- [Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics](https://mlsanity.com/paper/2106.02749/) (2021)
- [On Expressivity and Trainability of Quadratic Networks](https://mlsanity.com/paper/2110.06081/) (2021)
- [Long Sequence Hopfield Memory](https://mlsanity.com/paper/2306.04532/) (2023)
- [A Person Re-identification Data Augmentation Method with Adversarial Defense Effect](https://mlsanity.com/paper/2101.08783/) (2021)
- [Sparse Distributed Memory is a Continual Learner](https://mlsanity.com/paper/2303.11934/) (2023)
- [Dense Associative Memory with Epanechnikov Energy](https://mlsanity.com/paper/2506.10801/) (2025)
- [Dense Hebbian neural networks: a replica symmetric picture of unsupervised learning](https://mlsanity.com/paper/2211.14067/) (2022)
- [Dense Hebbian neural networks: a replica symmetric picture of supervised learning](https://mlsanity.com/paper/2212.00606/) (2022)
- [Modern Hopfield Networks Require Chain-of-Thought to Solve mathsfNC1-Hard Problems](https://mlsanity.com/paper/2412.05562/) (2024)
- [Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks](https://mlsanity.com/paper/2301.09245/) (2023)
- [Sequence Learning Using Equilibrium Propagation](https://mlsanity.com/paper/2209.09626/) (2022)
- [Hierarchical Associative Memory, Parallelized MLP-Mixer, and Symmetry Breaking](https://mlsanity.com/paper/2406.12220/) (2024)
- [On the Relationship Between Variational Inference and Auto-Associative Memory](https://mlsanity.com/paper/2210.08013/) (2022)
- [Dense Hopfield Networks in the Teacher-Student Setting](https://mlsanity.com/paper/2401.04191/) (2024)
- [Deterministic versus stochastic dynamical classifiers: opposing random adversarial attacks with noise](https://mlsanity.com/paper/2409.13470/) (2024)
- [iMixer: hierarchical Hopfield network implies an invertible, implicit and iterative MLP-Mixer](https://mlsanity.com/paper/2304.13061/) (2023)
- [Hebbian Learning from First Principles](https://mlsanity.com/paper/2401.07110/) (2024)
- [Prototype Analysis in Hopfield Networks with Hebbian Learning](https://mlsanity.com/paper/2407.03342/) (2024)
- [A Waddington landscape for prototype learning in generalized Hopfield networks](https://mlsanity.com/paper/2312.03012/) (2023)
- [Dense associative memory for Gaussian distributions](https://mlsanity.com/paper/2509.23162/) (2025)
- [Deep Clustering with Associative Memories](https://mlsanity.com/paper/2601.00963/) (2026)
- [A brain basis of dynamical intelligence for AI and computational neuroscience](https://mlsanity.com/paper/2105.07284/) (2021)
- [Improving Neural ODEs via Knowledge Distillation](https://mlsanity.com/paper/2203.05103/) (2022)
- [A Theory of Natural Intelligence](https://mlsanity.com/paper/2205.00002/) (2022)

## Abstract (arXiv)

Deep neural networks (DNN) trained in a supervised way suffer from two known problems. First, the minima of the objective function used in learning correspond to data points (also known as rubbish examples or fooling images) that lack semantic similarity with the training data. Second, a clean input can be changed by a small, and often imperceptible for human vision, perturbation, so that the resulting deformed input is misclassified by the network. These findings emphasize the differences between the ways DNN and humans classify patterns, and raise a question of designing learning algorithms that more accurately mimic human perception compared to the existing methods. Our paper examines these questions within the framework of Dense Associative Memory (DAM) models. These models are defined by the energy function, with higher order (higher than quadratic) interactions between the neurons. We show that in the limit when the power of the interaction vertex in the energy function is sufficiently large, these models have the following three properties. First, the minima of the objective function are free from rubbish images, so that each minimum is a semantically meaningful pattern. Second, artificial patterns poised precisely at the decision boundary look ambiguous to human subjects and share aspects of both classes that are separated by that decision boundary. Third, adversarial images constructed by models with small power of the interaction vertex, which are equivalent to DNN with rectified linear units (ReLU), fail to transfer to and fool the models with higher order interactions. This opens up a possibility to use higher order models for detecting and stopping malicious adversarial attacks. The presented results suggest that DAM with higher order energy functions are closer to human visual perception than DNN with ReLUs.