# Biological learning in key-value memory networks

arXiv: https://arxiv.org/abs/2110.13976 · HTML: https://mlsanity.com/paper/2110.13976/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2021.

## Built on by (most cited first, 18 of 18)

- [Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models](https://mlsanity.com/paper/2404.03827/) (2024)
- [CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory](https://mlsanity.com/paper/2402.13449/) (2024)
- [Key-value memory in the brain](https://mlsanity.com/paper/2501.02950/) (2025)
- [Simplicial Hopfield networks](https://mlsanity.com/paper/2305.05179/) (2023)
- [Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models](https://mlsanity.com/paper/2309.16750/) (2023)
- [Sparse and Structured Hopfield Networks](https://mlsanity.com/paper/2402.13725/) (2024)
- [Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffold](https://mlsanity.com/paper/2202.00159/) (2022)
- [Hebbian and Gradient-based Plasticity Enables Robust Memory and Rapid Learning in RNNs](https://mlsanity.com/paper/2302.03235/) (2023)
- [An In-depth Investigation of Sparse Rate Reduction in Transformer-like Models](https://mlsanity.com/paper/2411.17182/) (2024)
- [Associative memory inspires improvements for in-context learning using a novel attention residual stream architecture](https://mlsanity.com/paper/2412.15113/) (2024)
- [Semantically-correlated memories in a dense associative model](https://mlsanity.com/paper/2404.07123/) (2024)
- [From Memories to Maps: Mechanisms of In-Context Reinforcement Learning in Transformers](https://mlsanity.com/paper/2506.19686/) (2025)
- [GENESIS: A Generative Model of Episodic-Semantic Interaction](https://mlsanity.com/paper/2510.15828/) (2025)
- [On the Role of Hidden States of Modern Hopfield Network in Transformer](https://mlsanity.com/paper/2511.20698/) (2025)
- [Modern Hopfield Networks meet Encoded Neural Representations -- Addressing Practical Considerations](https://mlsanity.com/paper/2409.16408/) (2024)
- [A Neural Model of Rule Discovery with Relatively Short-Term Sequence Memory](https://mlsanity.com/paper/2412.06839/) (2024)
- [Modern Hopfield Networks with Continuous-Time Memories](https://mlsanity.com/paper/2502.10122/) (2025)
- [Attention as Conditioning: What Classical Learning Theory Predicts About Linear Transformers](https://mlsanity.com/paper/2508.08289/) (2025)

## Abstract (arXiv)

In neuroscience, classical Hopfield networks are the standard biologically plausible model of long-term memory, relying on Hebbian plasticity for storage and attractor dynamics for recall. In contrast, memory-augmented neural networks in machine learning commonly use a key-value mechanism to store and read out memories in a single step. Such augmented networks achieve impressive feats of memory compared to traditional variants, yet their biological relevance is unclear. We propose an implementation of basic key-value memory that stores inputs using a combination of biologically plausible three-factor plasticity rules. The same rules are recovered when network parameters are meta-learned. Our network performs on par with classical Hopfield networks on autoassociative memory tasks and can be naturally extended to continual recall, heteroassociative memory, and sequence learning. Our results suggest a compelling alternative to the classical Hopfield network as a model of biological long-term memory.