# Peripheral Vision Transformer

arXiv: https://arxiv.org/abs/2206.06801 · HTML: https://mlsanity.com/paper/2206.06801/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 67 of its 93 references that are in the map.
- First posted on arXiv 14 Jun 2022.

## Builds on (most cited first, 30 of 67)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [CoAtNet: Marrying Convolution and Attention for All Data Sizes](https://mlsanity.com/paper/2106.04803/) (2021)

## Built on by (most cited first, 27 of 27)

- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity](https://mlsanity.com/paper/2207.03620/) (2022)
- [TransNeXt: Robust Foveal Visual Perception for Vision Transformers](https://mlsanity.com/paper/2311.17132/) (2023)
- [SG-Former: Self-guided Transformer with Evolving Token Reallocation](https://mlsanity.com/paper/2308.12216/) (2023)
- [HiFuse: Hierarchical Multi-Scale Feature Fusion Network for Medical Image Classification](https://mlsanity.com/paper/2209.10218/) (2022)
- [PeLK: Parameter-efficient Large Kernel ConvNets with Peripheral Convolution](https://mlsanity.com/paper/2403.07589/) (2024)
- [EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm](https://mlsanity.com/paper/2206.09325/) (2022)
- [Lightweight Vision Transformer with Bidirectional Interaction](https://mlsanity.com/paper/2306.00396/) (2023)
- [OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels](https://mlsanity.com/paper/2502.20087/) (2025)
- [DAT++: Spatially Dynamic Vision Transformer with Deformable Attention](https://mlsanity.com/paper/2309.01430/) (2023)
- [GLH-Water: A Large-Scale Dataset for Global Surface Water Detection in Large-Size Very-High-Resolution Satellite Imagery](https://mlsanity.com/paper/2303.09310/) (2023)
- [A Dual-Stream Neural Network Explains the Functional Segregation of Dorsal and Ventral Visual Pathways in Human Brains](https://mlsanity.com/paper/2310.13849/) (2023)
- [Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers](https://mlsanity.com/paper/2507.15833/) (2025)
- [LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate](https://mlsanity.com/paper/2405.13985/) (2024)
- [3D Geometric Shape Assembly via Efficient Point Cloud Matching](https://mlsanity.com/paper/2407.10542/) (2024)
- [LLMind: Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Models](https://mlsanity.com/paper/2603.14882/) (2026)
- [Rethinking Local Perception in Lightweight Vision Transformer](https://mlsanity.com/paper/2303.17803/) (2023)
- [Segment This Thing: Foveated Tokenization for Efficient Point-Prompted Segmentation](https://mlsanity.com/paper/2506.11131/) (2025)
- [Human Vision Based 3D Point Cloud Semantic Segmentation of Large-Scale Outdoor Scene](https://mlsanity.com/paper/2301.12972/) (2023)
- [Progressive Learning with Cross-Window Consistency for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2211.12425/) (2022)
- [Foveated Reasoning: Stateful, Action-based Visual Focusing for Vision-Language Models](https://mlsanity.com/paper/2604.21079/) (2026)
- [EHSNet: End-to-End Holistic Learning Network for Large-Size Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2211.11316/) (2022)
- [SEDA: A Self-Adapted Entity-Centric Data Augmentation for Boosting Gird-based Discontinuous NER Models](https://mlsanity.com/paper/2511.20143/) (2025)
- [PFGNet: A Fully Convolutional Frequency-Guided Peripheral Gating Network for Efficient Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2602.20537/) (2026)
- [Deep Hierarchical Knowledge Loss for Fault Intensity Diagnosis](https://mlsanity.com/paper/2604.16459/) (2026)
- [Advancing Vision Transformer with Enhanced Spatial Priors](https://mlsanity.com/paper/2604.18549/) (2026)
- [Entropy-Guided Self-Supervised Learning for Medical Image Classification](https://mlsanity.com/paper/2605.21970/) (2026)

## Abstract (arXiv)

Human vision possesses a special type of visual processing systems called peripheral vision. Partitioning the entire visual field into multiple contour regions based on the distance to the center of our gaze, the peripheral vision provides us the ability to perceive various visual features at different regions. In this work, we take a biologically inspired approach and explore to model peripheral vision in deep neural networks for visual recognition. We propose to incorporate peripheral position encoding to the multi-head self-attention layers to let the network learn to partition the visual field into diverse peripheral regions given training data. We evaluate the proposed network, dubbed PerViT, on ImageNet-1K and systematically investigate the inner workings of the model for machine perception, showing that the network learns to perceive visual data similarly to the way that human vision does. The performance improvements in image classification over the baselines across different model sizes demonstrate the efficacy of the proposed method.