mlsanityData to 14 Oct 2026

Foveation-Guided Dynamic Token Selection for Robust and Efficient Vision Transformers

arXiv 2607.09480 ↗First posted 10 Jul 2026cs.CV cs.LG cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

The human visual system (HVS) employs foveated sampling and eye movements to achieve efficient perception, conserving both metabolic energy and computational resources. Drawing inspiration from this robustness and adaptability, we introduce the Foveated Dynamic Transformer (FDT), a foveation-guided dynamic token-selection architecture that integrates these mechanisms into a vision transformer framework. The FDT exhibits strong resilience to various types of noise and adversarial attacks, despite not being explicitly trained for such challenges. This inherent robustness is achieved through the use of fixation and foveation modules: the fixation module identifies fixation points to filter out irrelevant information, while the foveation module generates foveated embeddings with multi-scale information. At the 50% fixation-budget setting, FDT achieves higher accuracy than DeiT-S (81.9% vs. 80.9%) while reducing multiply-accumulate operations by 34.57%, highlighting one operating point on its accuracy-efficiency trade-off. These attributes position FDT as an HVS-inspired step toward artificial neural networks that combine adaptive computation with improved resilience.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 45 references that are in the map. First posted on arXiv 10 Jul 2026.

Builds on

21 of 45 references.

References · 21

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  5. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  6. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  7. 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
  8. 2020Quantifying Attention Flow in Transformerscited by 775
  9. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  10. 2020Torchattacks: A PyTorch Repository for Adversarial Attackscited by 158
  11. 2021Learned Token Pruning for Transformerscited by 139
  12. 2021Patch Slimming for Efficient Vision Transformerscited by 133
9 more
  1. 2021Adaptive Focus for Efficient Video Recognitioncited by 77
  2. 2018Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Networkcited by 69
  3. 2023X-Pruner: eXplainable Pruning for Vision Transformerscited by 68
  4. 2023Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformerscited by 62
  5. 2021FoveaTer: Foveated Transformer for Image Classificationcited by 28
  6. 2020Emergent Properties of Foveated Perceptual Systemscited by 24
  7. 2019Adversarial Robustness as a Prior for Learned Representationscited by 20
  8. 2025Efficient Token Compression for Vision Transformer with Spatial Information Preservedcited by 4
  9. 2025Gaze-Guided Learning: Avoiding Shortcut Bias in Visual Classificationcited by 4

Built on by

Not cited yet.