# On the role of feedback in visual processing: a predictive coding perspective

arXiv: https://arxiv.org/abs/2106.04225 · HTML: https://mlsanity.com/paper/2106.04225/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models](https://mlsanity.com/paper/1805.06605/) (2018)
- [MagNet: a Two-Pronged Defense against Adversarial Examples](https://mlsanity.com/paper/1705.09064/) (2017)
- [Learning long-range spatial dependencies with horizontal gated-recurrent units](https://mlsanity.com/paper/1805.08315/) (2018)
- [Recurrence is required to capture the representational dynamics of the human visual system](https://mlsanity.com/paper/1903.05946/) (2019)
- [APE-GAN: Adversarial Perturbation Elimination with GAN](https://mlsanity.com/paper/1707.05474/) (2017)
- [The Robust Manifold Defense: Adversarial Training using Generative Models](https://mlsanity.com/paper/1712.09196/) (2017)
- [Feedback Networks](https://mlsanity.com/paper/1612.09508/) (2016)
- [Task-Driven Convolutional Recurrent Models of the Visual System](https://mlsanity.com/paper/1807.00053/) (2018)
- [Deep Predictive Coding Network for Object Recognition](https://mlsanity.com/paper/1802.04762/) (2018)
- [Neural Networks with Recurrent Generative Feedback](https://mlsanity.com/paper/2007.09200/) (2020)
- [Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics](https://mlsanity.com/paper/2106.02749/) (2021)
- [Deep Predictive Coding Networks](https://mlsanity.com/paper/1301.3541/) (2013)
- [Predictive coding feedback results in perceived illusory contours in a recurrent neural network](https://mlsanity.com/paper/2102.01955/) (2021)
- [Recurrent Feedback Improves Feedforward Representations in Deep Neural Networks](https://mlsanity.com/paper/1912.10489/) (2019)
- [GAttANet: Global attention agreement for convolutional neural networks](https://mlsanity.com/paper/2104.05575/) (2021)

## Built on by (most cited first, 2 of 2)

- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [Deep Recurrence for Dynamical Segmentation Models](https://mlsanity.com/paper/2507.10143/) (2025)

## Abstract (arXiv)

Brain-inspired machine learning is gaining increasing consideration, particularly in computer vision. Several studies investigated the inclusion of top-down feedback connections in convolutional networks; however, it remains unclear how and when these connections are functionally helpful. Here we address this question in the context of object recognition under noisy conditions. We consider deep convolutional networks (CNNs) as models of feed-forward visual processing and implement Predictive Coding (PC) dynamics through feedback connections (predictive feedback) trained for reconstruction or classification of clean images. To directly assess the computational role of predictive feedback in various experimental situations, we optimize and interpret the hyper-parameters controlling the network's recurrent dynamics. That is, we let the optimization process determine whether top-down connections and predictive coding dynamics are functionally beneficial. Across different model depths and architectures (3-layer CNN, ResNet18, and EfficientNetB0) and against various types of noise (CIFAR100-C), we find that the network increasingly relies on top-down predictions as the noise level increases; in deeper networks, this effect is most prominent at lower layers. In addition, the accuracy of the network implementing PC dynamics significantly increases over time-steps, compared to its equivalent forward network. All in all, our results provide novel insights relevant to Neuroscience by confirming the computational role of feedback connections in sensory systems, and to Machine Learning by revealing how these can improve the robustness of current vision models.