# Classification and Reconstruction Processes in Deep Predictive Coding Networks: Antagonists or Allies?

arXiv: https://arxiv.org/abs/2401.09237 · HTML: https://mlsanity.com/paper/2401.09237/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 17 Jan 2024.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning](https://mlsanity.com/paper/1605.08104/) (2016)
- [Predictive Coding: a Theoretical and Experimental Review](https://mlsanity.com/paper/2107.12979/) (2021)
- [Comparing deep neural networks against humans: object recognition when the signal gets weaker](https://mlsanity.com/paper/1706.06969/) (2017)
- [Deep Predictive Coding Network with Local Recurrent Processing for Object Recognition](https://mlsanity.com/paper/1805.07526/) (2018)
- [Deep Predictive Coding Network for Object Recognition](https://mlsanity.com/paper/1802.04762/) (2018)
- [Neural Networks with Recurrent Generative Feedback](https://mlsanity.com/paper/2007.09200/) (2020)
- [Predictive Coding as a Neuromorphic Alternative to Backpropagation: A Critical Evaluation](https://mlsanity.com/paper/2304.02658/) (2023)
- [Preventing Deterioration of Classification Accuracy in Predictive Coding Networks](https://mlsanity.com/paper/2208.07114/) (2022)
- [Hierarchical Predictive Coding Models in a Deep-Learning Framework](https://mlsanity.com/paper/2005.03230/) (2020)

## Built on by (most cited first, 2 of 2)

- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [Understanding Transformer-based Vision Models through Inversion](https://mlsanity.com/paper/2412.06534/) (2024)

## Abstract (arXiv)

Predictive coding-inspired deep networks for visual computing integrate classification and reconstruction processes in shared intermediate layers. Although synergy between these processes is commonly assumed, it has yet to be convincingly demonstrated. In this study, we take a critical look at how classifying and reconstructing interact in deep learning architectures. Our approach utilizes a purposefully designed family of model architectures reminiscent of autoencoders, each equipped with an encoder, a decoder, and a classification head featuring varying modules and complexities. We meticulously analyze the extent to which classification- and reconstruction-driven information can seamlessly coexist within the shared latent layer of the model architectures. Our findings underscore a significant challenge: Classification-driven information diminishes reconstruction-driven information in intermediate layers' shared representations and vice versa. While expanding the shared representation's dimensions or increasing the network's complexity can alleviate this trade-off effect, our results challenge prevailing assumptions in predictive coding and offer guidance for future iterations of predictive coding concepts in deep networks.