# Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images

arXiv: https://arxiv.org/abs/2211.12047 · HTML: https://mlsanity.com/paper/2211.12047/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 69 references that are in the map.
- First posted on arXiv 22 Nov 2022.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016)
- [Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning](https://mlsanity.com/paper/1605.08104/) (2016)
- [Difference Target Propagation](https://mlsanity.com/paper/1412.7525/) (2014)
- [Decoupled Neural Interfaces using Synthetic Gradients](https://mlsanity.com/paper/1608.05343/) (2016)
- [Training End-to-End Analog Neural Networks with Equilibrium Propagation](https://mlsanity.com/paper/2006.01981/) (2020)
- [Random feedback weights support learning in deep neural networks](https://mlsanity.com/paper/1411.0247/) (2014)
- [The Neural Coding Framework for Learning Generative Models](https://mlsanity.com/paper/2012.03405/) (2020)
- [Associative Memories via Predictive Coding](https://mlsanity.com/paper/2109.08063/) (2021)
- [Biologically Motivated Algorithms for Propagating Local Target Representations](https://mlsanity.com/paper/1805.11703/) (2018)
- [Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules](https://mlsanity.com/paper/2006.16981/) (2020)
- [Deep Predictive Coding Network with Local Recurrent Processing for Object Recognition](https://mlsanity.com/paper/1805.07526/) (2018)
- [Deep Predictive Coding Network for Object Recognition](https://mlsanity.com/paper/1802.04762/) (2018)
- [Reverse Differentiation via Predictive Coding](https://mlsanity.com/paper/2103.04689/) (2021)
- [Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics](https://mlsanity.com/paper/2106.02749/) (2021)
- [Spiking Neural Predictive Coding for Continual Learning from Data Streams](https://mlsanity.com/paper/1908.08655/) (2019)
- [Deep Predictive Coding Networks](https://mlsanity.com/paper/1301.3541/) (2013)
- [Learning to Adapt by Minimizing Discrepancy](https://mlsanity.com/paper/1711.11542/) (2017)
- [Kickback cuts Backprop's red-tape: Biologically plausible credit assignment in neural networks](https://mlsanity.com/paper/1411.6191/) (2014)
- [Active Predictive Coding Networks: A Neural Solution to the Problem of Learning Reference Frames and Part-Whole Hierarchies](https://mlsanity.com/paper/2201.08813/) (2022)
- [Exploiting Spatio-Temporal Structure with Recurrent Winner-Take-All Networks](https://mlsanity.com/paper/1611.00050/) (2016)
- [An Empirical Analysis of Recurrent Learning Algorithms In Neural Lossy Image Compression Systems](https://mlsanity.com/paper/2201.11782/) (2022)

## Built on by (most cited first, 10 of 10)

- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment](https://mlsanity.com/paper/2312.09257/) (2023)
- [Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization](https://mlsanity.com/paper/2302.05440/) (2023)
- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [Active Sensing with Predictive Coding and Uncertainty Minimization](https://mlsanity.com/paper/2307.00668/) (2023)
- [Divide-and-Conquer Predictive Coding: a structured Bayesian inference algorithm](https://mlsanity.com/paper/2408.05834/) (2024)
- [Deep Predictive Coding with Bi-directional Propagation for Classification and Reconstruction](https://mlsanity.com/paper/2305.18472/) (2023)
- [Bridging Predictive Coding and MDL: A Two-Part Code Framework for Deep Learning](https://mlsanity.com/paper/2505.14635/) (2025)
- [Meta-Representational Predictive Coding: Neuroscience-Informed Self-Supervised Learning](https://mlsanity.com/paper/2503.21796/) (2025)
- [Comparison of Different Deep Neural Network Models in the Cultural Heritage Domain](https://mlsanity.com/paper/2504.21387/) (2025)

## Abstract (arXiv)

In this work, we develop convolutional neural generative coding (Conv-NGC), a generalization of predictive coding to the case of convolution/deconvolution-based computation. Specifically, we concretely implement a flexible neurobiologically-motivated algorithm that progressively refines latent state feature maps in order to dynamically form a more accurate internal representation/reconstruction model of natural images. The performance of the resulting sensory processing system is evaluated on complex datasets such as Color-MNIST, CIFAR-10, and Street House View Numbers (SVHN). We study the effectiveness of our brain-inspired model on the tasks of reconstruction and image denoising and find that it is competitive with convolutional auto-encoding systems trained by backpropagation of errors and outperforms them with respect to out-of-distribution reconstruction (including the full 90k CINIC-10 test set).