# On associative neural networks for sparse patterns with huge capacities

arXiv: https://arxiv.org/abs/2603.26217 · HTML: https://mlsanity.com/paper/2603.26217/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 26 references that are in the map.
- First posted on arXiv 27 Mar 2026.

## Builds on (most cited first, 5 of 5)

- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Large Associative Memory Problem in Neurobiology and Machine Learning](https://mlsanity.com/paper/2008.06996/) (2020)
- [In search of dispersed memories: Generative diffusion models are associative memory networks](https://mlsanity.com/paper/2309.17290/) (2023)
- [Memorization to Generalization: Emergence of Diffusion Models from Associative Memory](https://mlsanity.com/paper/2505.21777/) (2025)

## Abstract (arXiv)

Generalized Hopfield models with higher-order or exponential interaction terms are known to have substantially larger storage capacities than the classical quadratic model. On the other hand, associative memories for sparse patterns, such as the Willshaw and Amari models, already exhibit enhanced storage capacities in the sparse regime. In this paper we combine these two mechanisms. We introduce higher-order versions of sparse associative memory models and study their storage capacities in the sense of fixed-pattern stability. For the Amari and Willshaw models with fixed interaction order n, we obtain storage scales of order fracNn(log N)n. When the interaction order grows logarithmically with the number of neurons, the resulting storage scale becomes super-polynomial. We also study higher-order interactions in the block-structured Gripon--Berrou architecture, where the natural storage scale is of order cn. Our results show that the capacity increase caused by higher-order interactions persists in the sparse setting, while the precise storage scale depends on the underlying architecture.