mlsanityData to 7 Oct 2026

PatchDropout: Economizing Vision Transformers Using Patch Dropout

arXiv 2208.07220 ↗First posted 10 Aug 2022cs.CV cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Vision transformers have demonstrated the potential to outperform CNNs in a variety of vision tasks. But the computational and memory requirements of these models prohibit their use in many applications, especially those that depend on high-resolution images, such as medical image classification. Efforts to train ViTs more efficiently are overly complicated, necessitating architectural changes or intricate training schemes. In this work, we show that standard ViT models can be efficiently trained at high resolution by randomly dropping input image patches. This simple approach, PatchDropout, reduces FLOPs and memory by at least 50% in standard natural image datasets such as ImageNet, and those savings only increase with image size. On CSAW, a high-resolution medical dataset, we observe a 5 times savings in computation and memory using PatchDropout, along with a boost in performance. For practitioners with a fixed computational or memory budget, PatchDropout makes it possible to choose image resolution, hyperparameters, or model size to get the most performance out of their model.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 10 Aug 2022.

Builds on

27 of 35 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  10. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  11. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  12. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
15 more
  1. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  2. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  3. 2021Intriguing Properties of Vision Transformerscited by 446
  4. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
  5. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 359
  6. 2022Transformers in Medical Imaging: A Surveycited by 245
  7. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  8. 2021Patch Slimming for Efficient Vision Transformerscited by 131
  9. 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
  10. 2022Learning to Merge Tokens in Vision Transformerscited by 64
  11. 2022What Makes Transfer Learning Work For Medical Images: Feature Reuse & Other Factorscited by 61
  12. 2021Is it Time to Replace CNNs with Transformers for Medical Images?cited by 60
  13. 2021CSAW-M: An Ordinal Classification Dataset for Benchmarking Mammographic Masking of Cancercited by 9
  14. 2020Decoupling Inherent Risk and Early Cancer Signs in Image-based Breast Cancer Risk Modelscited by 5
  15. 2020Adding Seemingly Uninformative Labels Helps in Low Data Regimescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 3; 2025: 11; 2026: 6.

28 papers.

Sort:

By area:

  1. 2023Which Tokens to Use? Investigating Token Reduction in Vision Transformerscited by 59
  2. 2022Compute-Efficient Deep Learning: Algorithmic Trends and Opportunitiescited by 38
  3. 2022Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuningcited by 33
  4. 2023Learning Probabilistic Symmetrization for Architecture Agnostic Equivariancecited by 25
  5. 2023GTP-ViT: Efficient Vision Transformers via Graph-based Token Propagationcited by 19
  6. 2024Frozen Feature Augmentation for Few-Shot Image Classificationcited by 13
  7. 2025Transferable Adversarial Attacks on Black-Box Vision-Language Modelscited by 10
  8. 2026OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligencecited by 10
  9. 2025Sparse Imagination for Efficient Visual World Model Planningcited by 7
  10. 2025Thicker and Quicker: A Jumbo Token for Fast Plain Vision Transformerscited by 4
  11. 2023Beyond Grids: Exploring Elastic Input Sampling for Vision Transformerscited by 3
  12. 2023Geometry-Biased Transformer for Robust Multi-View 3D Human Pose Reconstructioncited by 2
16 more
  1. 2024Homogeneous Tokenizer Matters: Homogeneous Visual Tokenizer for Remote Sensing Image Understandingcited by 2
  2. 2025ForAug: Mitigating Biases in Image Classification via Controlled Image Compositionscited by 2
  3. 2025Video, How Do Your Tokens Merge?cited by 2
  4. 2025ResidualViT for Efficient Temporally Dense Video Encodingcited by 2
  5. 2025Towards Robust Real-World Multivariate Time Series Forecasting: A Unified Framework for Dependency, Asynchrony, and Missingnesscited by 1
  6. 2023Self-distilled Masked Attention guided masked image modeling with noise Regularized Teacher (SMART) for medical image analysiscited by 0
  7. 2024Random Token Fusion for Multi-View Medical Diagnosiscited by 0
  8. 2025PDC-ViT : Source Camera Identification using Pixel Difference Convolution and Vision Transformercited by 0
  9. 2025SPoT: Subpixel Placement of Tokens in Vision Transformerscited by 0
  10. 2025Neural Inhibition Improves Dynamic Routing and Mixture of Expertscited by 0
  11. 2025Efficient Multi-Source Knowledge Transfer by Model Mergingcited by 0
  12. 2026Effect of Patch Size on Fine-Tuning Vision Transformers in Two-Dimensional and Three-Dimensional Medical Image Classificationcited by 0
  13. 2026Variational Kernel Design for Internal Noise: Gaussian Chaos Noise, Representation Compatibility, and Reliable Deep Learningcited by 0
  14. 2026Critical Patch-Aware Sparse Prompting with Decoupled Training for Continual Learning on the Edgecited by 0
  15. 2026Co-distilled attention guided masked image modeling with noisy teacher for self-supervised learning on medical imagescited by 0
  16. 2026CoVisco: Codec-Native Vision Encoder with Native Token Compression for Unified Image-Video Understandingcited by 0