# Towards Robust Image Classification Using Sequential Attention Models

arXiv: https://arxiv.org/abs/1912.02184 · HTML: https://mlsanity.com/paper/1912.02184/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2019.

## Built on by (most cited first, 27 of 27)

- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Improving Robustness using Generated Data](https://mlsanity.com/paper/2110.09468/) (2021)
- [Visual Attention Methods in Deep Learning: An In-Depth Survey](https://mlsanity.com/paper/2204.07756/) (2022)
- [Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classification](https://mlsanity.com/paper/2101.06635/) (2021)
- [CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection](https://mlsanity.com/paper/2102.05311/) (2021)
- [Reinforced Attention for Few-Shot Learning and Beyond](https://mlsanity.com/paper/2104.04192/) (2021)
- [Feature Separation and Recalibration for Adversarial Robustness](https://mlsanity.com/paper/2303.13846/) (2023)
- [Rethinking Natural Adversarial Examples for Classification Models](https://mlsanity.com/paper/2102.11731/) (2021)
- [CAD: Co-Adapting Discriminative Features for Improved Few-Shot Classification](https://mlsanity.com/paper/2203.13465/) (2022)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [DSNet: A Dual-Stream Framework for Weakly-Supervised Gigapixel Pathology Image Analysis](https://mlsanity.com/paper/2109.05788/) (2021)
- [Impact of Attention on Adversarial Robustness of Image Classification Models](https://mlsanity.com/paper/2109.00936/) (2021)
- [Human Eyes Inspired Recurrent Neural Networks are More Robust Against Adversarial Noises](https://mlsanity.com/paper/2206.07282/) (2022)
- [Visual Attention Emerges from Recurrent Sparse Reconstruction](https://mlsanity.com/paper/2204.10962/) (2022)
- [MALICE: Manipulation Attacks on Learned Image ComprEssion](https://mlsanity.com/paper/2205.13253/) (2022)
- [Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition](https://mlsanity.com/paper/2110.04954/) (2021)
- [Perception Over Time: Temporal Dynamics for Robust Image Understanding](https://mlsanity.com/paper/2203.06254/) (2022)
- [Modeling Human Eye Movements with Neural Networks in a Maze-Solving Task](https://mlsanity.com/paper/2212.10367/) (2022)
- [Robust Image Classification: Defensive Strategies against FGSM and PGD Adversarial Attacks](https://mlsanity.com/paper/2408.13274/) (2024)
- [Test-Time Defense Against Adversarial Attacks via Stochastic Resonance of Latent Ensembles](https://mlsanity.com/paper/2510.03224/) (2025)
- [Bio-inspired Robustness: A Review](https://mlsanity.com/paper/2103.09265/) (2021)
- [Multi-Glimpse Network: A Robust and Efficient Classification Architecture based on Recurrent Downsampled Attention](https://mlsanity.com/paper/2111.02018/) (2021)
- [Hard Attention Control By Mutual Information Maximization](https://mlsanity.com/paper/2103.06371/) (2021)
- [Stationary Point Losses for Robust Model](https://mlsanity.com/paper/2302.09575/) (2023)
- [Enhancing Robust Representation in Adversarial Training: Alignment and Exclusion Criteria](https://mlsanity.com/paper/2310.03358/) (2023)
- [Harmonizing Feature Maps: A Graph Convolutional Approach for Enhancing Adversarial Robustness](https://mlsanity.com/paper/2406.11576/) (2024)
- [Fusing Foveal Fixations Using Linear Retinal Transformations and Bayesian Experimental Design](https://mlsanity.com/paper/2505.01249/) (2025)

## Abstract (arXiv)

In this paper we propose to augment a modern neural-network architecture with an attention model inspired by human perception. Specifically, we adversarially train and analyze a neural model incorporating a human inspired, visual attention component that is guided by a recurrent top-down sequential process. Our experimental evaluation uncovers several notable findings about the robustness and behavior of this new model. First, introducing attention to the model significantly improves adversarial robustness resulting in state-of-the-art ImageNet accuracies under a wide range of random targeted attack strengths. Second, we show that by varying the number of attention steps (glances/fixations) for which the model is unrolled, we are able to make its defense capabilities stronger, even in light of stronger attacks --- resulting in a "computational race" between the attacker and the defender. Finally, we show that some of the adversarial examples generated by attacking our model are quite different from conventional adversarial examples --- they contain global, salient and spatially coherent structures coming from the target class that would be recognizable even to a human, and work by distracting the attention of the model away from the main object in the original image.