# Differentiable Generalised Predictive Coding

arXiv: https://arxiv.org/abs/2112.03378 · HTML: https://mlsanity.com/paper/2112.03378/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning](https://mlsanity.com/paper/1605.08104/) (2016)
- [Predictive Coding Approximates Backprop along Arbitrary Computation Graphs](https://mlsanity.com/paper/2006.04182/) (2020)
- [Predictive Coding: a Theoretical and Experimental Review](https://mlsanity.com/paper/2107.12979/) (2021)
- [Deep active inference agents using Monte-Carlo methods](https://mlsanity.com/paper/2006.04176/) (2020)
- [Deep Active Inference as Variational Policy Gradients](https://mlsanity.com/paper/1907.03876/) (2019)
- [Deep Active Inference](https://mlsanity.com/paper/1709.02341/) (2017)
- [A free energy principle for a particular physics](https://mlsanity.com/paper/1906.10184/) (2019)
- [Predictive Coding, Variational Autoencoders, and Biological Connections](https://mlsanity.com/paper/2011.07464/) (2020)
- [Deep Active Inference for Partially Observable MDPs](https://mlsanity.com/paper/2009.03622/) (2020)
- [On the relationship between predictive coding and backpropagation](https://mlsanity.com/paper/2106.13082/) (2021)
- [Learning Perception and Planning with Deep Active Inference](https://mlsanity.com/paper/2001.11841/) (2020)
- [Markov Blankets in the Brain](https://mlsanity.com/paper/2006.02741/) (2020)
- [PredProp: Bidirectional Stochastic Optimization with Precision Weighted Predictive Coding](https://mlsanity.com/paper/2111.08792/) (2021)
- [Predictive coding, precision and natural gradients](https://mlsanity.com/paper/2111.06942/) (2021)

## Abstract (arXiv)

This paper deals with differentiable dynamical models congruent with neural process theories that cast brain function as the hierarchical refinement of an internal generative model explaining observations. Our work extends existing implementations of gradient-based predictive coding with automatic differentiation and allows to integrate deep neural networks for non-linear state parameterization. Gradient-based predictive coding optimises inferred states and weights locally in for each layer by optimising precision-weighted prediction errors that propagate from stimuli towards latent states. Predictions flow backwards, from latent states towards lower layers. The model suggested here optimises hierarchical and dynamical predictions of latent states. Hierarchical predictions encode expected content and hierarchical structure. Dynamical predictions capture changes in the encoded content along with higher order derivatives. Hierarchical and dynamical predictions interact and address different aspects of the same latent states. We apply the model to various perception and planning tasks on sequential data and show their mutual dependence. In particular, we demonstrate how learning sampling distances in parallel address meaningful locations data sampled at discrete time steps. We discuss possibilities to relax the assumption of linear hierarchies in favor of more flexible graph structure with emergent properties. We compare the granular structure of the model with canonical microcircuits describing predictive coding in biological networks and review the connection to Markov Blankets as a tool to characterize modularity. A final section sketches out ideas for efficient perception and planning in nested spatio-temporal hierarchies.