mlsanityData to 7 Oct 2026

FlexiViT: One Model for All Patch Sizes

arXiv 2212.08013 ↗First posted 15 Dec 2022cs.CV cs.AI cs.LGcited by 151

Not on the map. Why?

Abstract (arXiv)

Vision Transformers convert images to sequences by slicing them into patches. The size of these patches controls a speed/accuracy tradeoff, with smaller patches leading to higher accuracy at greater computational cost, but changing the patch size typically requires retraining the model. In this paper, we demonstrate that simply randomizing the patch size at training time leads to a single set of weights that performs well across a wide range of patch sizes, making it possible to tailor the model to different compute budgets at deployment time. We extensively evaluate the resulting model, which we call FlexiViT, on a wide range of tasks, including classification, image-text retrieval, open-world detection, panoptic segmentation, and semantic segmentation, concluding that it usually matches, and sometimes outperforms, standard ViT models trained at a single patch size in an otherwise identical setup. Hence, FlexiViT training is a simple drop-in improvement for ViT that makes it easy to add compute-adaptive capabilities to most models relying on a ViT backbone architecture. Code and pre-trained models are available at https://github.com/google-research/big_vision

Key facts. Cited by 151 papers in the map (data through 7 Oct 2026). Builds on 59 of its 83 references that are in the map. First posted on arXiv 15 Dec 2022.

Builds on

59 of 83 references.

References · 59

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2016Layer Normalizationcited by 5,579
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  11. 2022A ConvNet for the 2020scited by 3,545
  12. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
38 more
  1. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  2. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  3. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  4. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  5. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  6. 2019Natural Adversarial Examplescited by 1,401
  7. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  8. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  9. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  10. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  11. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  12. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  13. 2021Scaling Vision Transformerscited by 993
  14. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  15. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  16. 2018Panoptic Segmentationcited by 796
  17. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  18. 2021Going deeper with Image Transformerscited by 759
  19. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  20. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  21. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  22. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
  23. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  24. 2022DeiT III: Revenge of the ViTcited by 417
  25. 2019On the Efficacy of Knowledge Distillationcited by 377
  26. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
  27. 2020Are we done with ImageNet?cited by 329
  28. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 303
  29. 2021Knowledge distillation: A good teacher is patient and consistentcited by 255
  30. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
  31. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
  32. 2022Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Expertscited by 202
  33. 2020BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Modelscited by 187
  34. 2021Does Knowledge Distillation Really Work?cited by 187
  35. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  36. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  37. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  38. 2022ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Modelscited by 141

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 26; 2024: 40; 2025: 45; 2026: 40.

151 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  3. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  4. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 226
  5. 2024Rotary Position Embedding for Vision Transformercited by 226
  6. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 157
  7. 2024PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognitioncited by 141
  8. 2024Multimodal Autoregressive Pre-training of Large Vision Encoderscited by 108
  9. 2024RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Modelscited by 88
  10. 2023Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Designcited by 87
  11. 2023MatFormer: Nested Transformer for Elastic Inferencecited by 79
  12. 2023RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Drivingcited by 76
38 more
  1. 2025RF-DETR: Neural Architecture Search for Real-Time Detection Transformerscited by 70
  2. 2025Your ViT is Secretly an Image Segmentation Modelcited by 61
  3. 2023Which Tokens to Use? Investigating Token Reduction in Vision Transformerscited by 59
  4. 2025Towards a Unified Copernicus Foundation Model for Earth Visioncited by 52
  5. 2024Don't Look Twice: Faster Video Transformers with Run-Length Tokenizationcited by 42
  6. 2025Galileo: Learning Global & Local Features of Many Remote Sensing Modalitiescited by 41
  7. 2024Adaptive Length Image Tokenization via Recurrent Allocationcited by 40
  8. 2024Data curation via joint example selection further accelerates multimodal learningcited by 32
  9. 2026LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligencecited by 31
  10. 2024Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Surveycited by 27
  11. 2023Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimizationcited by 26
  12. 2024ViTAR: Vision Transformer with Any Resolutioncited by 25
  13. 2025FlexiMo: A Flexible Remote Sensing Foundation Modelcited by 24
  14. 2023FFT-based Dynamic Token Mixer for Visioncited by 22
  15. 2025OlmoEarth: Stable Latent Image Modeling for Multimodal Earth Observationcited by 20
  16. 2023MSViT: Dynamic Mixed-Scale Tokenization for Vision Transformerscited by 19
  17. 2025Training state-of-the-art pathology foundation models with orders of magnitude less datacited by 19
  18. 2024Token Cropr: Faster ViTs for Quite a Few Taskscited by 18
  19. 2024PLUTO: Pathology-Universal Transformercited by 17
  20. 2024AMES: Asymmetric and Memory-Efficient Similarity Estimation for Instance-level Retrievalcited by 16
  21. 2024BA-SAM: Scalable Bias-Mode Attention Mask for Segment Anything Modelcited by 15
  22. 2024How to Benchmark Vision Foundation Models for Semantic Segmentation?cited by 15
  23. 2024Understanding Visual Feature Reliance through the Lens of Complexitycited by 15
  24. 2025A Survey of Pathology Foundation Model: Progress and Future Directionscited by 14
  25. 2025SapiensID: Foundation for Human Recognitioncited by 14
  26. 2023Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformerscited by 13
  27. 2024Agglomerative Token Clusteringcited by 13
  28. 2023Win-Win: Training High-Resolution Vision Transformers from Two Windowscited by 12
  29. 2024On Improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Modelscited by 12
  30. 2025FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Computecited by 12
  31. 2025Accelerating Vision Transformers with Adaptive Patch Sizescited by 12
  32. 2024Wavelet-Based Image Tokenizer for Vision Transformerscited by 10
  33. 2023Dual PatchNormcited by 9
  34. 2023SBCFormer: Lightweight Network Capable of Full-size ImageNet Classification at 1 FPS on Single Board Computerscited by 9
  35. 2024LookHere: Vision Transformers with Directed Attention Generalize and Extrapolatecited by 9
  36. 2024A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Modelscited by 9
  37. 2024HydraViT: Stacking Heads for a Scalable ViTcited by 9
  38. 2024AdaGlimpse: Active Visual Exploration with Arbitrary Glimpse Position and Scalecited by 7

50 most cited of 151 All 151 (JSON)