# Kernel Memory Networks: A Unifying Framework for Memory Modeling

arXiv: https://arxiv.org/abs/2208.09416 · HTML: https://mlsanity.com/paper/2208.09416/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2022.

## Built on by (most cited first, 12 of 12)

- [Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models](https://mlsanity.com/paper/2404.03827/) (2024)
- [Provably Optimal Memory Capacity for Modern Hopfield Models: Transformer-Compatible Dense Associative Memories as Spherical Codes](https://mlsanity.com/paper/2410.23126/) (2024)
- [Sequential Memory with Temporal Predictive Coding](https://mlsanity.com/paper/2305.11982/) (2023)
- [Nonparametric Modern Hopfield Models](https://mlsanity.com/paper/2404.03900/) (2024)
- [Dense Associative Memory Through the Lens of Random Features](https://mlsanity.com/paper/2410.24153/) (2024)
- [Kernel Logistic Regression Learning for High-Capacity Hopfield Networks](https://mlsanity.com/paper/2504.07633/) (2025)
- [The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks](https://mlsanity.com/paper/2306.16922/) (2023)
- [Kernel Ridge Regression for Efficient Learning of High-Capacity Hopfield Networks](https://mlsanity.com/paper/2504.12561/) (2025)
- [On the Role of Hidden States of Modern Hopfield Network in Transformer](https://mlsanity.com/paper/2511.20698/) (2025)
- [Lifelong In-Context Learning with Transformers Requires Parametric Forms of Attention](https://mlsanity.com/paper/2606.25342/) (2026)
- [Modern Hopfield Networks meet Encoded Neural Representations -- Addressing Practical Considerations](https://mlsanity.com/paper/2409.16408/) (2024)
- [DeltaNet-PC: Test-Time Memory Without Backpropagation Through Time](https://mlsanity.com/paper/2610.15185/) (2026)

## Abstract (arXiv)

We consider the problem of training a neural network to store a set of patterns with maximal noise robustness. A solution, in terms of optimal weights and state update rules, is derived by training each individual neuron to perform either kernel classification or interpolation with a minimum weight norm. By applying this method to feed-forward and recurrent networks, we derive optimal models, termed kernel memory networks, that include, as special cases, many of the hetero- and auto-associative memory models that have been proposed over the past years, such as modern Hopfield networks and Kanerva's sparse distributed memory. We modify Kanerva's model and demonstrate a simple way to design a kernel memory network that can store an exponential number of continuous-valued patterns with a finite basin of attraction. The framework of kernel memory networks offers a simple and intuitive way to understand the storage capacity of previous memory models, and allows for new biological interpretations in terms of dendritic non-linearities and synaptic cross-talk.