# Foveation in the Era of Deep Learning

arXiv: https://arxiv.org/abs/2312.01450 · HTML: https://mlsanity.com/paper/2312.01450/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 3 Dec 2023.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Recurrent Models of Visual Attention](https://mlsanity.com/paper/1406.6247/) (2014)
- [Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs](https://mlsanity.com/paper/1704.02901/) (2017)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [Polar Transformer Networks](https://mlsanity.com/paper/1709.01889/) (2017)
- [Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds](https://mlsanity.com/paper/1806.01759/) (2018)
- [ConvPoint: Continuous Convolutions for Point Cloud Processing](https://mlsanity.com/paper/1904.02375/) (2019)
- [Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks](https://mlsanity.com/paper/1809.03355/) (2018)
- [Saccader: Improving Accuracy of Hard Attention Models for Vision](https://mlsanity.com/paper/1908.07644/) (2019)
- [Structured Receptive Fields in CNNs](https://mlsanity.com/paper/1605.02971/) (2016)
- [FOVEA: Foveated Image Magnification for Autonomous Navigation](https://mlsanity.com/paper/2108.12102/) (2021)
- [FoveaTer: Foveated Transformer for Image Classification](https://mlsanity.com/paper/2105.14173/) (2021)
- [Scale-covariant and scale-invariant Gaussian derivative networks](https://mlsanity.com/paper/2011.14759/) (2020)
- [Dynamic Computational Time for Visual Attention](https://mlsanity.com/paper/1703.10332/) (2017)
- [Attention for Fine-Grained Categorization](https://mlsanity.com/paper/1412.7054/) (2014)
- [CyCNN: A Rotation Invariant CNN using Polar Mapping and Cylindrical Convolution Layers](https://mlsanity.com/paper/2007.10588/) (2020)
- [FoveaNet: Perspective-aware Urban Scene Parsing](https://mlsanity.com/paper/1708.02421/) (2017)
- [Non-discriminative data or weak model? On the relative importance of data and model resolution](https://mlsanity.com/paper/1909.03205/) (2019)
- [Consistency driven Sequential Transformers Attention Model for Partially Observable Scenes](https://mlsanity.com/paper/2204.00656/) (2022)
- [Emergence of foveal image sampling from learning to attend in visual scenes](https://mlsanity.com/paper/1611.09430/) (2016)

## Built on by (most cited first, 8 of 8)

- [Eye, Robot: Learning to Look to Act with a BC-RL Perception-Action Loop](https://mlsanity.com/paper/2506.10968/) (2025)
- [Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers](https://mlsanity.com/paper/2507.15833/) (2025)
- [LLMind: Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Models](https://mlsanity.com/paper/2603.14882/) (2026)
- [Foveated Diffusion: Efficient Spatially Adaptive Image and Video Generation](https://mlsanity.com/paper/2603.23491/) (2026)
- [FOVI: A biologically-inspired foveated interface for deep vision models](https://mlsanity.com/paper/2602.03766/) (2026)
- [Policy-based Foveated Imaging and Perception](https://mlsanity.com/paper/2606.02565/) (2026)
- [HiResNets: Native Full-HD Video Recognition with Foveal Residual Streams](https://mlsanity.com/paper/2608.02140/) (2026)
- [Adaptive Observation Control for Active Visual Recognition](https://mlsanity.com/paper/2610.19624/) (2026)

## Abstract (arXiv)

In this paper, we tackle the challenge of actively attending to visual scenes using a foveated sensor. We introduce an end-to-end differentiable foveated active vision architecture that leverages a graph convolutional network to process foveated images, and a simple yet effective formulation for foveated image sampling. Our model learns to iteratively attend to regions of the image relevant for classification. We conduct detailed experiments on a variety of image datasets, comparing the performance of our method with previous approaches to foveated vision while measuring how the impact of different choices, such as the degree of foveation, and the number of fixations the network performs, affect object recognition performance. We find that our model outperforms a state-of-the-art CNN and foveated vision architectures of comparable parameters and a given pixel or computation budget