# BayesPCN: A Continually Learnable Predictive Coding Associative Memory

arXiv: https://arxiv.org/abs/2205.09930 · HTML: https://mlsanity.com/paper/2205.09930/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 20 May 2022.

## Builds on (most cited first, 27 of 27)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [Linear Transformers Are Secretly Fast Weight Programmers](https://mlsanity.com/paper/2102.11174/) (2021)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Large Associative Memory Problem in Neurobiology and Machine Learning](https://mlsanity.com/paper/2008.06996/) (2020)
- [Predictive Coding Approximates Backprop along Arbitrary Computation Graphs](https://mlsanity.com/paper/2006.04182/) (2020)
- [Predictive Coding: a Theoretical and Experimental Review](https://mlsanity.com/paper/2107.12979/) (2021)
- [Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models](https://mlsanity.com/paper/2202.04557/) (2022)
- [Modern Hopfield Networks and Attention for Immune Repertoire Classification](https://mlsanity.com/paper/2007.13505/) (2020)
- [The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention](https://mlsanity.com/paper/2202.05798/) (2022)
- [The Neural Coding Framework for Learning Generative Models](https://mlsanity.com/paper/2012.03405/) (2020)
- [Associative Memories via Predictive Coding](https://mlsanity.com/paper/2109.08063/) (2021)
- [Hierarchical Associative Memory](https://mlsanity.com/paper/2107.06446/) (2021)
- [Unsupervised Progressive Learning and the STAM Architecture](https://mlsanity.com/paper/1904.02021/) (2019)
- [Attention Approximates Sparse Distributed Memory](https://mlsanity.com/paper/2111.05498/) (2021)
- [Hybrid Predictive Coding: Inferring, Fast and Slow](https://mlsanity.com/paper/2204.02169/) (2022)
- [Continuous online sequence learning with an unsupervised neural network model](https://mlsanity.com/paper/1512.05463/) (2015)
- [Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting](https://mlsanity.com/paper/1905.10696/) (2019)
- [The Kanerva Machine: A Generative Distributed Memory](https://mlsanity.com/paper/1804.01756/) (2018)
- [Modern Hopfield Networks for Few- and Zero-Shot Reaction Template Prediction](https://mlsanity.com/paper/2104.03279/) (2021)
- [Conducting Credit Assignment by Aligning Local Representations](https://mlsanity.com/paper/1803.01834/) (2018)
- [Meta-Learning Deep Energy-Based Memory Models](https://mlsanity.com/paper/1910.02720/) (2019)
- [Content Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffold](https://mlsanity.com/paper/2202.00159/) (2022)
- [Learning Attractor Dynamics for Generative Memory](https://mlsanity.com/paper/1811.09556/) (2018)
- [A neuro-inspired architecture for unsupervised continual learning based on online clustering and hierarchical predictive coding](https://mlsanity.com/paper/1810.09391/) (2018)

## Built on by (most cited first, 14 of 14)

- [Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models](https://mlsanity.com/paper/2404.03827/) (2024)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Benchmarking Predictive Coding Networks -- Made Simple](https://mlsanity.com/paper/2407.01163/) (2024)
- [Sequential Memory with Temporal Predictive Coding](https://mlsanity.com/paper/2305.11982/) (2023)
- [Predictive Coding beyond Correlations](https://mlsanity.com/paper/2306.15479/) (2023)
- [A Sparse Quantized Hopfield Network for Online-Continual Memory](https://mlsanity.com/paper/2307.15040/) (2023)
- [Saliency-Guided Hidden Associative Replay for Continual Learning](https://mlsanity.com/paper/2310.04334/) (2023)
- [Faster Predictive Coding Networks via Better Initialization](https://mlsanity.com/paper/2601.20895/) (2026)
- [Associative Memories in the Feature Space](https://mlsanity.com/paper/2402.10814/) (2024)
- [Lifelong Learning of Video Diffusion Models From a Single Video Stream](https://mlsanity.com/paper/2406.04814/) (2024)
- [Memory-Integrated Reconfigurable Adapters: A Unified Framework for Settings with Multiple Tasks](https://mlsanity.com/paper/2512.00940/) (2025)
- [Predictive Attractor Models](https://mlsanity.com/paper/2410.02430/) (2024)
- [Robust Auto-associative Memory via Convolutional Restricted Hopfield Networks](https://mlsanity.com/paper/2606.20666/) (2026)
- [DeltaNet-PC: Test-Time Memory Without Backpropagation Through Time](https://mlsanity.com/paper/2610.15185/) (2026)

## Abstract (arXiv)

Associative memory plays an important role in human intelligence and its mechanisms have been linked to attention in machine learning. While the machine learning community's interest in associative memories has recently been rekindled, most work has focused on memory recall (read) over memory learning (write). In this paper, we present BayesPCN, a hierarchical associative memory capable of performing continual one-shot memory writes without meta-learning. Moreover, BayesPCN is able to gradually forget past observations (forget) to free its memory. Experiments show that BayesPCN can recall corrupted i.i.d. high-dimensional data observed hundreds to a thousand ``timesteps'' ago without a large drop in recall ability compared to the state-of-the-art offline-learned parametric memory models.