# The Neural Coding Framework for Learning Generative Models

arXiv: https://arxiv.org/abs/2012.03405 · HTML: https://mlsanity.com/paper/2012.03405/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Dec 2020.

## Built on by (most cited first, 30 of 44)

- [Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?](https://mlsanity.com/paper/2202.09467/) (2022)
- [Associative Memories via Predictive Coding](https://mlsanity.com/paper/2109.08063/) (2021)
- [In search of dispersed memories: Generative diffusion models are associative memory networks](https://mlsanity.com/paper/2309.17290/) (2023)
- [World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challenges](https://mlsanity.com/paper/2301.05832/) (2023)
- [The Predictive Forward-Forward Algorithm](https://mlsanity.com/paper/2301.01452/) (2023)
- [Learning on Arbitrary Graph Topologies via Predictive Coding](https://mlsanity.com/paper/2201.13180/) (2022)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Benchmarking Predictive Coding Networks -- Made Simple](https://mlsanity.com/paper/2407.01163/) (2024)
- [A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks](https://mlsanity.com/paper/2212.00720/) (2022)
- [The Cascaded Forward Algorithm for Neural Network Training](https://mlsanity.com/paper/2303.09728/) (2023)
- [Predictive Coding beyond Gaussian Distributions](https://mlsanity.com/paper/2211.03481/) (2022)
- [BayesPCN: A Continually Learnable Predictive Coding Associative Memory](https://mlsanity.com/paper/2205.09930/) (2022)
- [How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective](https://mlsanity.com/paper/2106.08453/) (2021)
- [Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization](https://mlsanity.com/paper/2302.05440/) (2023)
- [Backprop-Free Reinforcement Learning with Active Neural Generative Coding](https://mlsanity.com/paper/2107.07046/) (2021)
- [Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images](https://mlsanity.com/paper/2211.12047/) (2022)
- [A Mathematical Framework for Learning Probability Distributions](https://mlsanity.com/paper/2212.11481/) (2022)
- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [Towards the Training of Deeper Predictive Coding Neural Networks](https://mlsanity.com/paper/2506.23800/) (2025)
- [Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems](https://mlsanity.com/paper/2209.09174/) (2022)
- [Predictive Coding beyond Correlations](https://mlsanity.com/paper/2306.15479/) (2023)
- [A Review of Neuroscience-Inspired Machine Learning](https://mlsanity.com/paper/2403.18929/) (2024)
- [Robust Graph Representation Learning via Predictive Coding](https://mlsanity.com/paper/2212.04656/) (2022)
- [Are Biological Systems More Intelligent Than Artificial Intelligence?](https://mlsanity.com/paper/2405.02325/) (2024)
- [Maze Learning using a Hyperdimensional Predictive Processing Cognitive Architecture](https://mlsanity.com/paper/2204.00619/) (2022)
- [Neural Sampling in Hierarchical Exponential-family Energy-based Models](https://mlsanity.com/paper/2310.08431/) (2023)
- [Divide-and-Conquer Predictive Coding: a structured Bayesian inference algorithm](https://mlsanity.com/paper/2408.05834/) (2024)
- [Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networks](https://mlsanity.com/paper/2410.04708/) (2024)
- [Temporal-Difference Learning Using Distributed Error Signals](https://mlsanity.com/paper/2411.03604/) (2024)
- [Faster Predictive Coding Networks via Better Initialization](https://mlsanity.com/paper/2601.20895/) (2026)

## Abstract (arXiv)

Neural generative models can be used to learn complex probability distributions from data, to sample from them, and to produce probability density estimates. We propose a computational framework for developing neural generative models inspired by the theory of predictive processing in the brain. According to predictive processing theory, the neurons in the brain form a hierarchy in which neurons in one level form expectations about sensory inputs from another level. These neurons update their local models based on differences between their expectations and the observed signals. In a similar way, artificial neurons in our generative models predict what neighboring neurons will do, and adjust their parameters based on how well the predictions matched reality. In this work, we show that the neural generative models learned within our framework perform well in practice across several benchmark datasets and metrics and either remain competitive with or significantly outperform other generative models with similar functionality (such as the variational auto-encoder).