# On the use of Cortical Magnification and Saccades as Biological Proxies for Data Augmentation

arXiv: https://arxiv.org/abs/2112.07173 · HTML: https://mlsanity.com/paper/2112.07173/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 14 Dec 2021.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [A Self-supervised Approach for Adversarial Robustness](https://mlsanity.com/paper/2006.04924/) (2020)
- [Revisiting Active Perception](https://mlsanity.com/paper/1603.02729/) (2016)
- [Self-supervised learning through the eyes of a child](https://mlsanity.com/paper/2007.16189/) (2020)
- [Foveation-based Mechanisms Alleviate Adversarial Examples](https://mlsanity.com/paper/1511.06292/) (2015)
- [Biologically Inspired Mechanisms for Adversarial Robustness](https://mlsanity.com/paper/2006.16427/) (2020)
- [On the surprising similarities between supervised and self-supervised models](https://mlsanity.com/paper/2010.08377/) (2020)
- [FoveaTer: Foveated Transformer for Image Classification](https://mlsanity.com/paper/2105.14173/) (2021)
- [Emergent Properties of Foveated Perceptual Systems](https://mlsanity.com/paper/2006.07991/) (2020)
- [Object Detection Through Exploration With A Foveated Visual Field](https://mlsanity.com/paper/1408.0814/) (2014)
- [Towards Metamerism via Foveated Style Transfer](https://mlsanity.com/paper/1705.10041/) (2017)
- [FastSal: a Computationally Efficient Network for Visual Saliency Prediction](https://mlsanity.com/paper/2008.11151/) (2020)
- [Can Peripheral Representations Improve Clutter Metrics on Complex Scenes?](https://mlsanity.com/paper/1608.04042/) (2016)
- [Human peripheral blur is optimal for object recognition](https://mlsanity.com/paper/1807.08476/) (2018)
- [Computational role of eccentricity dependent cortical magnification](https://mlsanity.com/paper/1406.1770/) (2014)

## Built on by (most cited first, 12 of 12)

- [Time to augment self-supervised visual representation learning](https://mlsanity.com/paper/2207.13492/) (2022)
- [A Dual-Stream Neural Network Explains the Functional Segregation of Dorsal and Ventral Visual Pathways in Human Brains](https://mlsanity.com/paper/2310.13849/) (2023)
- [Learning Object Semantic Similarity with Self-Supervision](https://mlsanity.com/paper/2405.05143/) (2024)
- [Human Eyes Inspired Recurrent Neural Networks are More Robust Against Adversarial Noises](https://mlsanity.com/paper/2206.07282/) (2022)
- [FOVI: A biologically-inspired foveated interface for deep vision models](https://mlsanity.com/paper/2602.03766/) (2026)
- [Advancing Perception in Artificial Intelligence through Principles of Cognitive Science](https://mlsanity.com/paper/2310.08803/) (2023)
- [Human Gaze Boosts Object-Centered Representation Learning](https://mlsanity.com/paper/2501.02966/) (2025)
- [Embodied vision for learning object representations](https://mlsanity.com/paper/2205.06198/) (2022)
- [Toddlers' Active Gaze Behavior Supports Self-Supervised Object Learning](https://mlsanity.com/paper/2411.01969/) (2024)
- [Simulated Cortical Magnification Supports Self-Supervised Object Learning](https://mlsanity.com/paper/2509.15751/) (2025)
- [SAFER-AiD: Saccade-Assisted Foveal-peripheral vision Enhanced Reconstruction for Adversarial Defense](https://mlsanity.com/paper/2510.08761/) (2025)
- [Temporal Slowness in Central Vision Drives Semantic Object Learning](https://mlsanity.com/paper/2602.04462/) (2026)

## Abstract (arXiv)

Self-supervised learning is a powerful way to learn useful representations from natural data. It has also been suggested as one possible means of building visual representation in humans, but the specific objective and algorithm are unknown. Currently, most self-supervised methods encourage the system to learn an invariant representation of different transformations of the same image in contrast to those of other images. However, such transformations are generally non-biologically plausible, and often consist of contrived perceptual schemes such as random cropping and color jittering. In this paper, we attempt to reverse-engineer these augmentations to be more biologically or perceptually plausible while still conferring the same benefits for encouraging robust representation. Critically, we find that random cropping can be substituted by cortical magnification, and saccade-like sampling of the image could also assist the representation learning. The feasibility of these transformations suggests a potential way that biological visual systems could implement self-supervision. Further, they break the widely accepted spatially-uniform processing assumption used in many computer vision algorithms, suggesting a role for spatially-adaptive computation in humans and machines alike. Our code and demo can be found here.