# Saccader: Improving Accuracy of Hard Attention Models for Vision

arXiv: https://arxiv.org/abs/1908.07644 · HTML: https://mlsanity.com/paper/1908.07644/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2019.

## Built on by (most cited first, 30 of 35)

- [Differentiable Patch Selection for Image Recognition](https://mlsanity.com/paper/2104.03059/) (2021)
- [Vision Transformer with Progressive Sampling](https://mlsanity.com/paper/2108.01684/) (2021)
- [This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagation](https://mlsanity.com/paper/2108.12204/) (2021)
- [Interpretable by Design: Learning Predictors by Composing Interpretable Queries](https://mlsanity.com/paper/2207.00938/) (2022)
- [Hard-Attention for Scalable Image Classification](https://mlsanity.com/paper/2102.10212/) (2021)
- [AOE-Net: Entities Interactions Modeling with Adaptive Attention Mechanism for Temporal Action Proposals Generation](https://mlsanity.com/paper/2210.02578/) (2022)
- [Internally Rewarded Reinforcement Learning](https://mlsanity.com/paper/2302.00270/) (2023)
- [A Probabilistic Hard Attention Model For Sequentially Observed Scenes](https://mlsanity.com/paper/2111.07534/) (2021)
- [AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation](https://mlsanity.com/paper/2110.11474/) (2021)
- [Consistency driven Sequential Transformers Attention Model for Partially Observable Scenes](https://mlsanity.com/paper/2204.00656/) (2022)
- [Foveation in the Era of Deep Learning](https://mlsanity.com/paper/2312.01450/) (2023)
- [AdaGlimpse: Active Visual Exploration with Arbitrary Glimpse Position and Scale](https://mlsanity.com/paper/2404.03482/) (2024)
- [A Lightweight, Efficient and Explainable-by-Design Convolutional Neural Network for Internet Traffic Classification](https://mlsanity.com/paper/2202.05535/) (2022)
- [GQHAN: A Grover-inspired Quantum Hard Attention Network](https://mlsanity.com/paper/2401.14089/) (2024)
- [Mask or Non-Mask? Robust Face Mask Detector via Triplet-Consistency Representation Learning](https://mlsanity.com/paper/2110.00523/) (2021)
- [Towards Two-Stream Foveation-based Active Vision Learning](https://mlsanity.com/paper/2403.15977/) (2024)
- [Emergence of Fixational and Saccadic Movements in a Multi-Level Recurrent Attention Model for Vision](https://mlsanity.com/paper/2505.13191/) (2025)
- [Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition](https://mlsanity.com/paper/2110.04954/) (2021)
- [FOVI: A biologically-inspired foveated interface for deep vision models](https://mlsanity.com/paper/2602.03766/) (2026)
- [GliTr: Glimpse Transformers with Spatiotemporal Consistency for Online Action Prediction](https://mlsanity.com/paper/2210.13605/) (2022)
- [Semantic-Syntactic Discrepancy in Images (SSDI): Learning Meaning and Order of Features from Natural Images](https://mlsanity.com/paper/2401.17515/) (2024)
- [On Inherent Adversarial Robustness of Active Vision Systems](https://mlsanity.com/paper/2404.00185/) (2024)
- [Semantic-Based Active Perception for Humanoid Visual Tasks with Foveal Sensors](https://mlsanity.com/paper/2404.10836/) (2024)
- [TRIP: Trainable Region-of-Interest Prediction for Hardware-Efficient Neuromorphic Processing on Event-based Vision](https://mlsanity.com/paper/2406.17483/) (2024)
- [CanViT: Toward Active-Vision Foundation Models](https://mlsanity.com/paper/2603.22570/) (2026)
- [Adaptive sampling for scanning pixel cameras](https://mlsanity.com/paper/2207.13460/) (2022)
- [Energy-Efficient Visual Search by Eye Movement and Low-Latency Spiking Neural Network](https://mlsanity.com/paper/2310.06578/) (2023)
- [Language-Guided Reinforcement Learning for Hard Attention in Few-Shot Learning](https://mlsanity.com/paper/2310.07800/) (2023)
- [A Sensorimotor Vision Transformer](https://mlsanity.com/paper/2504.02536/) (2025)
- [SAFER-AiD: Saccade-Assisted Foveal-peripheral vision Enhanced Reconstruction for Adversarial Defense](https://mlsanity.com/paper/2510.08761/) (2025)

## Abstract (arXiv)

Although deep convolutional neural networks achieve state-of-the-art performance across nearly all image classification tasks, their decisions are difficult to interpret. One approach that offers some level of interpretability by design is textithard attention, which uses only relevant portions of the image. However, training hard attention models with only class label supervision is challenging, and hard attention has proved difficult to scale to complex datasets. Here, we propose a novel hard attention model, which we term Saccader. Key to Saccader is a pretraining step that requires only class labels and provides initial attention locations for policy gradient optimization. Our best models narrow the gap to common ImageNet baselines, achieving 75% top-1 and 91% top-5 while attending to less than one-third of the image.