mlsanityData to 14 Oct 2026

Key-value memory in the brain

arXiv 2501.02950 ↗First posted 6 Jan 2025q-bio.NC cs.AI cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Classical models of memory in psychology and neuroscience rely on similarity-based retrieval of stored patterns, where similarity is a function of retrieval cues and the stored patterns. While parsimonious, these models do not allow distinct representations for storage and retrieval, despite their distinct computational demands. Key-value memory systems, in contrast, distinguish representations used for storage (values) and those used for retrieval (keys). This allows key-value memory systems to optimize simultaneously for fidelity in storage and discriminability in retrieval. We review the computational foundations of key-value memory, its role in modern machine learning systems, related ideas from psychology and neuroscience, applications to a number of empirical puzzles, and possible biological implementations.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 22 of its 139 references that are in the map. First posted on arXiv 6 Jan 2025.

Builds on

22 of 139 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  5. 2020Rethinking Attention with Performerscited by 1,601
  6. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  7. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  8. 2020Hopfield Networks is All You Needcited by 475
  9. 2021Random Feature Attentioncited by 354
  10. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 270
  11. 2016Dense Associative Memory for Pattern Recognitioncited by 236
  12. 2016Key-Value Memory Networks for Directly Reading Documentscited by 223
10 more
  1. 2019Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernelcited by 213
  2. 2016Using Fast Weights to Attend to the Recent Pastcited by 170
  3. 2021Relating transformers to models and neural representations of the hippocampal formationcited by 62
  4. 2020Learning to Encode Position for Transformer with Continuous Dynamical Modelcited by 61
  5. 2022Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Modelscited by 55
  6. 2022The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attentioncited by 46
  7. 2019Overparameterized Neural Networks Implement Associative Memorycited by 45
  8. 2021Biological learning in key-value memory networkscited by 18
  9. 2022Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffoldcited by 11
  10. 2023Sparse Distributed Memory is a Continual Learnercited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 11.

24 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2025Liger: Linearizing Large Language Models to Gated Recurrent Structurescited by 33
  3. 2025Comba: Improving Bilinear RNNs with Closed-loop Controlcited by 15
  4. 2025Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiencescited by 15
  5. 2026Fast-weight Product Key Memorycited by 13
  6. 2025Fast weight programming and linear transformers: from machine learning to neurobiologycited by 8
  7. 2025Flexible Prefrontal Control over Hippocampal Episodic Memory for Goal-Directed Generalizationcited by 4
  8. 2025Memorization and Knowledge Injection in Gated LLMscited by 3
  9. 2025A Scenario-Driven Cognitive Approach to Next-Generation AI Memorycited by 3
  10. 2026Sharp Capacity Thresholds in Linear Associative Memory: From Top-1 Retrieval to Tail-Average Learningcited by 3
  11. 2026Sparse Attention as Compact Kernel Regressioncited by 2
  12. 2025From Memories to Maps: Mechanisms of In-Context Reinforcement Learning in Transformerscited by 1
12 more
  1. 2025GENESIS: A Generative Model of Episodic-Semantic Interactioncited by 1
  2. 2026Why the Brain Consolidates: Predictive Forgetting for Optimal Generalisationcited by 1
  3. 2026HKVM-RAG: Key-Value-Separated Hypergraph Evidence Organization for Multi-Hop RAGcited by 1
  4. 2025Hebbian learning the local structure of languagecited by 0
  5. 2025Echo State Transformer: Attention Over Finite Memoriescited by 0
  6. 2025Dissociating model architectures from inference computationscited by 0
  7. 2026Memory Retrieval in Transformers: Insights from The Encoding Specificity Principlecited by 0
  8. 2026MemDefrag: Latent Memory Defragmentation for Large Language Modelscited by 0
  9. 2026Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuningcited by 0
  10. 2026Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Modelscited by 0
  11. 2026A neural network that maintains and retrieves memories based on contextcited by 0
  12. 2026Retrieval-Centric Deep Learning in Growing Nonparametric Neural Networkscited by 0