# Deep Predictive Coding Networks

arXiv: https://arxiv.org/abs/1301.3541 · HTML: https://mlsanity.com/paper/1301.3541/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2013.

## Built on by (most cited first, 13 of 13)

- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics](https://mlsanity.com/paper/2106.02749/) (2021)
- [Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment](https://mlsanity.com/paper/2312.09257/) (2023)
- [Predictive coding feedback results in perceived illusory contours in a recurrent neural network](https://mlsanity.com/paper/2102.01955/) (2021)
- [Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images](https://mlsanity.com/paper/2211.12047/) (2022)
- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations](https://mlsanity.com/paper/2101.06848/) (2021)
- [On the role of feedback in visual processing: a predictive coding perspective](https://mlsanity.com/paper/2106.04225/) (2021)
- [Midway Network: Learning Representations for Recognition and Motion from Latent Dynamics](https://mlsanity.com/paper/2510.05558/) (2025)
- [Metalearning: Sparse Variable-Structure Automata](https://mlsanity.com/paper/2102.00315/) (2021)
- [Fast Deep Predictive Coding Networks for Videos Feature Extraction without Labels](https://mlsanity.com/paper/2409.04945/) (2024)
- [Meta-Representational Predictive Coding: Neuroscience-Informed Self-Supervised Learning](https://mlsanity.com/paper/2503.21796/) (2025)
- [DMPCN: Dynamic Modulated Predictive Coding Network with Hybrid Feedback Representations](https://mlsanity.com/paper/2504.14665/) (2025)

## Abstract (arXiv)

The quality of data representation in deep learning methods is directly related to the prior model imposed on the representations; however, generally used fixed priors are not capable of adjusting to the context in the data. To address this issue, we propose deep predictive coding networks, a hierarchical generative model that empirically alters priors on the latent representations in a dynamic and context-sensitive manner. This model captures the temporal dependencies in time-varying signals and uses top-down information to modulate the representation in lower layers. The centerpiece of our model is a novel procedure to infer sparse states of a dynamic model which is used for feature extraction. We also extend this feature extraction block to introduce a pooling function that captures locally invariant representations. When applied on a natural video data, we show that our method is able to learn high-level visual features. We also demonstrate the role of the top-down connections by showing the robustness of the proposed model to structured noise.