mlsanityData to 7 Oct 2026

ViTAR: Vision Transformer with Any Resolution

arXiv 2403.18361 ↗First posted 27 Mar 2024cs.CVcited by 25

Not on the map. Why?

Abstract (arXiv)

This paper tackles a significant challenge faced by Vision Transformers (ViTs): their constrained scalability across different image resolutions. Typically, ViTs experience a performance decline when processing resolutions different from those seen during training. Our work introduces two key innovations to address this issue. Firstly, we propose a novel module for dynamic resolution adjustment, designed with a single Transformer block, specifically to achieve highly efficient incremental token integration. Secondly, we introduce fuzzy positional encoding in the Vision Transformer to provide consistent positional awareness across multiple resolutions, thereby preventing overfitting to any single training resolution. Our resulting model, ViTAR (Vision Transformer with Any Resolution), demonstrates impressive adaptability, achieving 83.3% top-1 accuracy at a 1120x1120 resolution and 80.4% accuracy at a 4032x4032 resolution, all while reducing computational costs. ViTAR also shows strong performance in downstream tasks such as instance and semantic segmentation and can easily combined with self-supervised learning techniques like Masked AutoEncoder. Our work provides a cost-effective solution for enhancing the resolution scalability of ViTs, paving the way for more versatile and efficient high-resolution image processing.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 34 of its 38 references that are in the map. First posted on arXiv 27 Mar 2024.

Builds on

34 of 38 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2017Mask R-CNNcited by 7,259
  9. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  12. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
22 more
  1. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  2. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  3. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  4. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  5. 2017Random Erasing Data Augmentationcited by 1,235
  6. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  7. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  8. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  9. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  10. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  11. 2021CMT: Convolutional Neural Networks Meet Vision Transformerscited by 310
  12. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 226
  13. 2022DaViT: Dual Attention Vision Transformerscited by 174
  14. 2022FlexiViT: One Model for All Patch Sizescited by 151
  15. 2021All Tokens Matter: Token Labeling for Training Better Vision Transformerscited by 144
  16. 2023Randomized Positional Encodings Boost Length Generalization of Transformerscited by 114
  17. 2022Inception Transformercited by 99
  18. 2022SVFormer: Semi-supervised Video Transformer for Action Recognitioncited by 67
  19. 2022Vision Transformer with Super Token Samplingcited by 43
  20. 2022ResFormer: Scaling ViTs with Multi-Resolution Trainingcited by 34
  21. 2023Lightweight Vision Transformer with Bidirectional Interactioncited by 25
  22. 2023DropPos: Pre-Training Vision Transformers by Reconstructing Dropped Positionscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 9; 2026: 11.

25 papers.

Sort:

By area:

  1. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 157
  2. 2025FlexiMo: A Flexible Remote Sensing Foundation Modelcited by 24
  3. 2025EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolationcited by 19
  4. 2025RoPECraft: Training-Free Motion Transfer with Trajectory-Guided RoPE Optimization on Diffusion Transformerscited by 14
  5. 2025PathoHR: Breast Cancer Survival Prediction on High-Resolution Pathological Imagescited by 9
  6. 2024Cross-Domain Foundation Model Adaptation: Pioneering Computer Vision Models for Geophysical Data Analysiscited by 4
  7. 2024MATEY: multiscale adaptive foundation models for spatiotemporal physical systemscited by 4
  8. 2025UniViTAR: Unified Vision Transformer with Native Resolutioncited by 3
  9. 2025SeqPE: Transformer with Sequential Position Encodingcited by 2
  10. 2025Tile-Based ViT Inference with Visual-Cluster Priors for Zero-Shot Multi-Species Plant Identificationcited by 2
  11. 2026Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmarkcited by 2
  12. 2024Liveness Detection in Computer Vision: Transformer-based Self-Supervised Learning for Face Anti-Spoofingcited by 1
13 more
  1. 2024SDformerFlow: Spatiotemporal swin spikeformer for event-based optical flow estimationcited by 1
  2. 2026MuViT: Multi-Resolution Vision Transformers for Learning Across Scales in Microscopycited by 1
  3. 2026VibeToken: Scaling 1D Image Tokenizers and Autoregressive Models for Dynamic Resolution Generationscited by 1
  4. 2026G-ray: Ray-Level Relative Geometric Position Encoding in Multi-View Vision Transformers under Camera Heterogeneitycited by 1
  5. 2025Input Resolution Downsizing as a Compression Technique for Vision Deep Learning Systemscited by 0
  6. 2025StretchySnake: Flexible SSM Training Unlocks Action Recognition Across Spatio-Temporal Scalescited by 0
  7. 2026ELSA: Exact Linear-Scan Attention for Fast and Memory-Light Vision Transformerscited by 0
  8. 2026On What We Can Learn from Low-Resolution Datacited by 0
  9. 2026Weighted Reverse Convolution for Feature Upsamplingcited by 0
  10. 2026Active Spatial Guidance: Eliminating Injected Positional Mechanisms in Vision Transformerscited by 0
  11. 2026PixelUp: Zero-Shot Semantic Feature Upsampling for Fine-Grained Vision Taskscited by 0
  12. 2026Variable-Granularity Tokenization for High-Resolution Object Detectioncited by 0
  13. 2026ProgResViT: Progressive Resolution and Width for Adaptive Vision Transformerscited by 0