# Deep Networks with Internal Selective Attention through Feedback Connections

arXiv: https://arxiv.org/abs/1407.3068 · HTML: https://mlsanity.com/paper/1407.3068/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2014.

## Built on by (most cited first, 17 of 17)

- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Attention Mechanism in Neural Networks: Where it Comes and Where it Goes](https://mlsanity.com/paper/2204.13154/) (2022)
- [Granger-causal Attentive Mixtures of Experts: Learning Important Features with Neural Networks](https://mlsanity.com/paper/1802.02195/) (2018)
- [Understanding the computational demands underlying visual reasoning](https://mlsanity.com/paper/2108.03603/) (2021)
- [SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networks](https://mlsanity.com/paper/2112.03731/) (2021)
- [Neural Attention Models in Deep Learning: Survey and Taxonomy](https://mlsanity.com/paper/2112.05909/) (2021)
- [Improving Automated Visual Fault Detection by Combining a Biologically Plausible Model of Visual Attention with Deep Learning](https://mlsanity.com/paper/2102.06955/) (2021)
- [Lambda-Skip Connections: the architectural component that prevents Rank Collapse](https://mlsanity.com/paper/2410.10609/) (2024)
- [Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations](https://mlsanity.com/paper/2101.06848/) (2021)
- [Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition](https://mlsanity.com/paper/2110.04954/) (2021)
- [Nighttime Thermal Infrared Image Colorization with Feedback-based Object Appearance Learning](https://mlsanity.com/paper/2310.15688/) (2023)
- [Object Based Attention Through Internal Gating](https://mlsanity.com/paper/2106.04540/) (2021)
- [Understanding top-down attention using task-oriented ablation design](https://mlsanity.com/paper/2106.11339/) (2021)
- [Visual Sensation and Perception Computational Models for Deep Learning: State of the art, Challenges and Prospects](https://mlsanity.com/paper/2109.03391/) (2021)
- [RCMHA: Relative Convolutional Multi-Head Attention for Natural Language Modelling](https://mlsanity.com/paper/2308.03429/) (2023)
- [Unsupervised Network for Single Image Raindrop Removal](https://mlsanity.com/paper/2412.03019/) (2024)
- [MBInception: A new Multi-Block Inception Model for Enhancing Image Processing Efficiency](https://mlsanity.com/paper/2412.13703/) (2024)

## Abstract (arXiv)

Traditional convolutional neural networks (CNN) are stationary and feedforward. They neither change their parameters during evaluation nor use feedback from higher to lower layers. Real brains, however, do. So does our Deep Attention Selective Network (dasNet) architecture. DasNets feedback structure can dynamically alter its convolutional filter sensitivities during classification. It harnesses the power of sequential processing to improve classification performance, by allowing the network to iteratively focus its internal attention on some of its convolutional filters. Feedback is trained through direct policy search in a huge million-dimensional parameter space, through scalable natural evolution strategies (SNES). On the CIFAR-10 and CIFAR-100 datasets, dasNet outperforms the previous state-of-the-art model.