# Storing overlapping associative memories on latent manifolds in low-rank spiking networks

arXiv: https://arxiv.org/abs/2411.17485 · HTML: https://mlsanity.com/paper/2411.17485/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 47 references that are in the map.
- First posted on arXiv 26 Nov 2024.

## Builds on (most cited first, 10 of 10)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Neural population geometry: An approach for understanding biological and artificial neural networks](https://mlsanity.com/paper/2104.07059/) (2021)
- [Linking connectivity, dynamics and computations in low-rank recurrent neural networks](https://mlsanity.com/paper/1711.09672/) (2017)
- [Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity](https://mlsanity.com/paper/2107.04084/) (2021)
- [Robust computation with rhythmic spike patterns](https://mlsanity.com/paper/1901.07718/) (2019)
- [Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffold](https://mlsanity.com/paper/2202.00159/) (2022)
- [Shaping dynamics with multiple populations in low-rank recurrent networks](https://mlsanity.com/paper/2007.02062/) (2020)
- [Approximating nonlinear functions with latent boundaries in low-rank excitatory-inhibitory spiking networks](https://mlsanity.com/paper/2307.09334/) (2023)

## Abstract (arXiv)

Associative memory architectures such as the Hopfield network have long been important conceptual and theoretical models for neuroscience and artificial intelligence. However, translating these abstract models into spiking neural networks has been surprisingly difficult. Indeed, much previous work has been restricted to storing a small number of primarily non-overlapping memories in large networks, thereby limiting their scalability. Here, we revisit the associative memory problem in light of recent advances in understanding spike-based computation. Using a recently-established geometric framework, we show that the spiking activity for a large class of all-inhibitory networks is situated on a low-dimensional, convex, and piecewise-linear manifold, with dynamics that move along the manifold. We then map the associative memory problem onto these dynamics, and demonstrate how the vertices of a hypercubic manifold can be used to store stable, overlapping activity patterns with a direct correspondence to the original Hopfield model. We propose several learning rules, and demonstrate a linear scaling of the storage capacity with the number of neurons, as well as robust pattern completion abilities. Overall, this work serves as a case study to demonstrate the effectiveness of using a geometrical perspective to design dynamics on neural manifolds, with implications for neuroscience and machine learning.