# Simplicial Hopfield networks

arXiv: https://arxiv.org/abs/2305.05179 · HTML: https://mlsanity.com/paper/2305.05179/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 181 references that are in the map.
- First posted on arXiv 9 May 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Hypergraph Neural Networks](https://mlsanity.com/paper/1809.09401/) (2018)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Query-Key Normalization for Transformers](https://mlsanity.com/paper/2010.04245/) (2020)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoning](https://mlsanity.com/paper/2204.08770/) (2022)
- [Large Associative Memory Problem in Neurobiology and Machine Learning](https://mlsanity.com/paper/2008.06996/) (2020)
- [Simplicial Neural Networks](https://mlsanity.com/paper/2010.03633/) (2020)
- [Random Walks on Simplicial Complexes and the normalized Hodge 1-Laplacian](https://mlsanity.com/paper/1807.05044/) (2018)
- [Principled Simplicial Neural Networks for Trajectory Prediction](https://mlsanity.com/paper/2102.10058/) (2021)
- [Cell Complex Neural Networks](https://mlsanity.com/paper/2010.00743/) (2020)
- [Toward a realistic model of speech processing in the brain with self-supervised learning](https://mlsanity.com/paper/2206.01685/) (2022)
- [Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models](https://mlsanity.com/paper/2202.04557/) (2022)
- [Modern Hopfield Networks and Attention for Immune Repertoire Classification](https://mlsanity.com/paper/2007.13505/) (2020)
- [Biological learning in key-value memory networks](https://mlsanity.com/paper/2110.13976/) (2021)
- [Drawing Inspiration from Biological Dendrites to Empower Artificial Neural Networks](https://mlsanity.com/paper/2106.07490/) (2021)
- [Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffold](https://mlsanity.com/paper/2202.00159/) (2022)
- [Logic and the 2-Simplicial Transformer](https://mlsanity.com/paper/1909.00668/) (2019)
- [A model for structured information representation in neural networks](https://mlsanity.com/paper/1611.03698/) (2016)
- [On the mapping between Hopfield networks and Restricted Boltzmann Machines](https://mlsanity.com/paper/2101.11744/) (2021)
- [Robust exponential memory in Hopfield networks](https://mlsanity.com/paper/1411.4625/) (2014)
- [Neural coordination can be enhanced by occasional interruption of normal firing patterns: A self-optimizing spiking neural network model](https://mlsanity.com/paper/1409.0470/) (2014)
- [Extended temporal association memory by inhibitory Hebbian learning](https://mlsanity.com/paper/1809.05254/) (2018)
- [Detecting danger in gridworlds using Gromov's Link Condition](https://mlsanity.com/paper/2201.06274/) (2022)

## Built on by (most cited first, 16 of 16)

- [Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models](https://mlsanity.com/paper/2404.03827/) (2024)
- [Outlier-Efficient Hopfield Layers for Large Transformer-Based Models](https://mlsanity.com/paper/2404.03828/) (2024)
- [BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model](https://mlsanity.com/paper/2404.03830/) (2024)
- [Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codes](https://mlsanity.com/paper/2410.23126/) (2024)
- [Long Sequence Hopfield Memory](https://mlsanity.com/paper/2306.04532/) (2023)
- [End-to-end Differentiable Clustering with Associative Memories](https://mlsanity.com/paper/2306.03209/) (2023)
- [Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models](https://mlsanity.com/paper/2309.16750/) (2023)
- [Dense Associative Memory Through the Lens of Random Features](https://mlsanity.com/paper/2410.24153/) (2024)
- [Representing Edge Flows on Graphs via Sparse Cell Complexes](https://mlsanity.com/paper/2309.01632/) (2023)
- [Associative memory inspires improvements for in-context learning using a novel attention residual stream architecture](https://mlsanity.com/paper/2412.15113/) (2024)
- [Semantically-correlated memories in a dense associative model](https://mlsanity.com/paper/2404.07123/) (2024)
- [Hopfield-Enhanced Deep Neural Networks for Artifact-Resilient Brain State Decoding](https://mlsanity.com/paper/2311.03421/) (2023)
- [On the Role of Hidden States of Modern Hopfield Network in Transformer](https://mlsanity.com/paper/2511.20698/) (2025)
- [Energy-Based Dynamical Models for Neurocomputation, Learning, and Optimization](https://mlsanity.com/paper/2604.05042/) (2026)
- [Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamics](https://mlsanity.com/paper/2603.20656/) (2026)
- [Hyperbolic Neural Population Geometry Benefits Computation](https://mlsanity.com/paper/2606.10238/) (2026)

## Abstract (arXiv)

Hopfield networks are artificial neural networks which store memory patterns on the states of their neurons by choosing recurrent connection weights and update rules such that the energy landscape of the network forms attractors around the memories. How many stable, sufficiently-attracting memory patterns can we store in such a network using N neurons? The answer depends on the choice of weights and update rule. Inspired by setwise connectivity in biology, we extend Hopfield networks by adding setwise connections and embedding these connections in a simplicial complex. Simplicial complexes are higher dimensional analogues of graphs which naturally represent collections of pairwise and setwise relationships. We show that our simplicial Hopfield networks increase memory storage capacity. Surprisingly, even when connections are limited to a small random subset of equivalent size to an all-pairwise network, our networks still outperform their pairwise counterparts. Such scenarios include non-trivial simplicial topology. We also test analogous modern continuous Hopfield networks, offering a potentially promising avenue for improving the attention mechanism in Transformer models.