# Feedback Networks

arXiv: https://arxiv.org/abs/1612.09508 · HTML: https://mlsanity.com/paper/1612.09508/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Dec 2016.

## Built on by (most cited first, 29 of 29)

- [High-resolution Iterative Feedback Network for Camouflaged Object Detection](https://mlsanity.com/paper/2203.11624/) (2022)
- [Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions](https://mlsanity.com/paper/2106.05022/) (2021)
- [Hitchhiker's Guide to Super-Resolution: Introduction and Recent Advances](https://mlsanity.com/paper/2209.13131/) (2022)
- [Unsupervised Multi-View Object Segmentation Using Radiance Field Propagation](https://mlsanity.com/paper/2210.00489/) (2022)
- [Hard-Attention for Scalable Image Classification](https://mlsanity.com/paper/2102.10212/) (2021)
- [Towards the Unseen: Iterative Text Recognition by Distilling from Errors](https://mlsanity.com/paper/2107.12081/) (2021)
- [Deep Feedback Inverse Problem Solver](https://mlsanity.com/paper/2101.07719/) (2021)
- [Towards Interpretable Video Super-Resolution via Alternating Optimization](https://mlsanity.com/paper/2207.10765/) (2022)
- [SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networks](https://mlsanity.com/paper/2112.03731/) (2021)
- [Improving Anytime Prediction with Parallel Cascaded Networks and a Temporal-Difference Loss](https://mlsanity.com/paper/2102.09808/) (2021)
- [REQA: Coarse-to-fine Assessment of Image Quality to Alleviate the Range Effect](https://mlsanity.com/paper/2209.01760/) (2022)
- [Compact and Optimal Deep Learning with Recurrent Parameter Generators](https://mlsanity.com/paper/2107.07110/) (2021)
- [Scale-Aware Dynamic Network for Continuous-Scale Super-Resolution](https://mlsanity.com/paper/2110.15655/) (2021)
- [TDAM: Top-Down Attention Module for Contextually Guided Feature Selection in CNNs](https://mlsanity.com/paper/2111.13470/) (2021)
- [Auto-Compressing Networks](https://mlsanity.com/paper/2506.09714/) (2025)
- [Feedback Pyramid Attention Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2106.06966/) (2021)
- [Learning to Accelerate Approximate Methods for Solving Integer Programming via Early Fixing](https://mlsanity.com/paper/2207.02087/) (2022)
- [On the role of feedback in visual processing: a predictive coding perspective](https://mlsanity.com/paper/2106.04225/) (2021)
- [Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition](https://mlsanity.com/paper/2110.04954/) (2021)
- [Characterising representation dynamics in recurrent neural networks for object recognition](https://mlsanity.com/paper/2308.12435/) (2023)
- [Nighttime Thermal Infrared Image Colorization with Feedback-based Object Appearance Learning](https://mlsanity.com/paper/2310.15688/) (2023)
- [Collaborative Feedback Discriminative Propagation for Video Super-Resolution](https://mlsanity.com/paper/2404.04745/) (2024)
- [Multi-frame Joint Enhancement for Early Interlaced Videos](https://mlsanity.com/paper/2109.14151/) (2021)
- [RecycleNet: Latent Feature Recycling Leads to Iterative Decision Refinement](https://mlsanity.com/paper/2309.07513/) (2023)
- [Iterative Feedback Network for Unsupervised Point Cloud Registration](https://mlsanity.com/paper/2401.04357/) (2024)
- [Super-resolving Compressed Images via Parallel and Series Integration of Artifact Reduction and Resolution Enhancement](https://mlsanity.com/paper/2103.01698/) (2021)
- [Content-adaptive Representation Learning for Fast Image Super-resolution](https://mlsanity.com/paper/2105.09645/) (2021)
- [Analyze and Design Network Architectures by Recursion Formulas](https://mlsanity.com/paper/2108.08689/) (2021)
- [Unsupervised Network for Single Image Raindrop Removal](https://mlsanity.com/paper/2412.03019/) (2024)

## Abstract (arXiv)

Currently, the most successful learning models in computer vision are based on learning successive representations followed by a decision layer. This is usually actualized through feedforward multilayer neural networks, e.g. ConvNets, where each layer forms one of such successive representations. However, an alternative that can achieve the same goal is a feedback based approach in which the representation is formed in an iterative manner based on a feedback received from previous iteration's output. We establish that a feedback based approach has several fundamental advantages over feedforward: it enables making early predictions at the query time, its output naturally conforms to a hierarchical structure in the label space (e.g. a taxonomy), and it provides a new basis for Curriculum Learning. We observe that feedback networks develop a considerably different representation compared to feedforward counterparts, in line with the aforementioned advantages. We put forth a general feedback based learning architecture with the endpoint results on par or better than existing feedforward networks with the addition of the above advantages. We also investigate several mechanisms in feedback architectures (e.g. skip connections in time) and design choices (e.g. feedback length). We hope this study offers new perspectives in quest for more natural and practical learning models.