# Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting

arXiv: https://arxiv.org/abs/1905.10696 · HTML: https://mlsanity.com/paper/1905.10696/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2019.

## Built on by (most cited first, 15 of 15)

- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks](https://mlsanity.com/paper/2212.00720/) (2022)
- [BayesPCN: A Continually Learnable Predictive Coding Associative Memory](https://mlsanity.com/paper/2205.09930/) (2022)
- [Continual Competitive Memory: A Neural System for Online Task-Free Lifelong Learning](https://mlsanity.com/paper/2106.13300/) (2021)
- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [Continual Learning of Dynamical Systems with Competitive Federated Reservoir Computing](https://mlsanity.com/paper/2206.13336/) (2022)
- [Predictive Coding beyond Correlations](https://mlsanity.com/paper/2306.15479/) (2023)
- [A Neuro-mimetic Realization of the Common Model of Cognition via Hebbian Learning and Free Energy Minimization](https://mlsanity.com/paper/2310.15177/) (2023)
- [A Contrastive Symmetric Forward-Forward Algorithm (SFFA) for Continual Learning Tasks](https://mlsanity.com/paper/2409.07387/) (2024)
- [Deep Predictive Coding with Bi-directional Propagation for Classification and Reconstruction](https://mlsanity.com/paper/2305.18472/) (2023)
- [Neuro-mimetic Task-free Unsupervised Online Learning with Continual Self-Organizing Maps](https://mlsanity.com/paper/2402.12465/) (2024)
- [Bridging Predictive Coding and MDL: A Two-Part Code Framework for Deep Learning](https://mlsanity.com/paper/2505.14635/) (2025)
- [Predictive Attractor Models](https://mlsanity.com/paper/2410.02430/) (2024)
- [Class Incremental Continual Learning with Self-Organizing Maps and Synthetic Replay](https://mlsanity.com/paper/2508.21240/) (2025)
- [Error Highways: Scaling Predictive Coding to Very Deep Networks](https://mlsanity.com/paper/2606.22744/) (2026)

## Abstract (arXiv)

In lifelong learning systems based on artificial neural networks, one of the biggest obstacles is the inability to retain old knowledge as new information is encountered. This phenomenon is known as catastrophic forgetting. In this paper, we propose a new kind of connectionist architecture, the Sequential Neural Coding Network, that is robust to forgetting when learning from streams of data points and, unlike networks of today, does not learn via the popular back-propagation of errors. Grounded in the neurocognitive theory of predictive processing, our model adapts synapses in a biologically-plausible fashion while another neural system learns to direct and control this cortex-like structure, mimicking some of the task-executive control functionality of the basal ganglia. In our experiments, we demonstrate that our self-organizing system experiences significantly less forgetting compared to standard neural models, outperforming a swath of previously proposed methods, including rehearsal/data buffer-based methods, on both standard (SplitMNIST, Split Fashion MNIST, etc.) and custom benchmarks even though it is trained in a stream-like fashion. Our work offers evidence that emulating mechanisms in real neuronal systems, e.g., local learning, lateral competition, can yield new directions and possibilities for tackling the grand challenge of lifelong machine learning.