mlsanityData to 7 Oct 2026

ResFormer: Scaling ViTs with Multi-Resolution Training

arXiv 2212.00776 ↗First posted 1 Dec 2022cs.CVcited by 34

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) have achieved overwhelming success, yet they suffer from vulnerable resolution scalability, i.e., the performance drops drastically when presented with input resolutions that are unseen during training. We introduce, ResFormer, a framework that is built upon the seminal idea of multi-resolution training for improved performance on a wide spectrum of, mostly unseen, testing resolutions. In particular, ResFormer operates on replicated images of different resolutions and enforces a scale consistency loss to engage interactive information across different scales. More importantly, to alternate among varying resolutions effectively, especially novel ones in testing, we propose a global-local positional embedding strategy that changes smoothly conditioned on input sizes. We conduct extensive experiments for image classification on ImageNet. The results provide strong quantitative evidence that ResFormer has promising scaling abilities towards a wide range of resolutions. For instance, ResFormer-B-MR achieves a Top-1 accuracy of 75.86% and 81.72% when evaluated on relatively low and high resolutions respectively (i.e., 96 and 640), which are 48% and 7.49% better than DeiT-B. We also demonstrate, moreover, ResFormer is flexible and can be easily extended to semantic segmentation, object detection and video action recognition. Code is available at https://github.com/ruitian12/resformer.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 77 of its 83 references that are in the map. First posted on arXiv 1 Dec 2022.

Builds on

77 of 83 references.

References · 77

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2020End-to-End Object Detection with Transformerscited by 7,384
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2017Mask R-CNNcited by 7,259
  12. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
38 more
  1. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  2. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  5. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  7. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  8. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  10. 2017The Kinetics Human Action Video Datasetcited by 1,894
  11. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  12. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  13. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  14. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  15. 2019Natural Adversarial Examplescited by 1,401
  16. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  17. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  18. 2017Random Erasing Data Augmentationcited by 1,235
  19. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  20. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  21. 2018Self-Attention with Relative Position Representationscited by 1,112
  22. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  23. 2021Video Swin Transformercited by 1,090
  24. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  25. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  26. 2021Scaling Vision Transformerscited by 993
  27. 2020Designing Network Design Spacescited by 954
  28. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  29. 2021Multiscale Vision Transformerscited by 896
  30. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  31. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  32. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  33. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  34. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  35. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  36. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 719
  37. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  38. 2022Vision Transformer Adapter for Dense Predictionscited by 564

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 12; 2025: 11; 2026: 6.

34 papers.

Sort:

By area:

  1. 2024Rotary Position Embedding for Vision Transformercited by 226
  2. 2024Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiTcited by 130
  3. 2024OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generationcited by 117
  4. 2022Prototypical Residual Networks for Anomaly Detection and Localizationcited by 56
  5. 2023Implicit Temporal Modeling with Learnable Alignment for Video Recognitioncited by 44
  6. 2024ViTAR: Vision Transformer with Any Resolutioncited by 25
  7. 2024OmniVid: A Generative Framework for Universal Video Understandingcited by 22
  8. 2025EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolationcited by 19
  9. 2023Win-Win: Training High-Resolution Vision Transformers from Two Windowscited by 12
  10. 2023Three Towers: Flexible Contrastive Learning with Pretrained Image Modelscited by 11
  11. 2024MSPE: Multi-Scale Patch Embedding Prompts Vision Transformers to Any Resolutioncited by 6
  12. 2025Make Your Training Flexible: Towards Deployment-Efficient Video Modelscited by 6
22 more
  1. 2025Enhancing Vision Foundation Models via Multimodal Continual Pre-Trainingcited by 5
  2. 2024Cross-Domain Foundation Model Adaptation: Pioneering Computer Vision Models for Geophysical Data Analysiscited by 4
  3. 2024BCTR: Bidirectional Conditioning Transformer for Scene Graph Generationcited by 3
  4. 2024From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensingcited by 3
  5. 2025Local Scale Equivariance with Latent Deep Equilibrium Canonicalizercited by 2
  6. 2026Cross-Scale Pansharpening via ScaleFormer and the PanScale Benchmarkcited by 2
  7. 2023DyRA: Portable Dynamic Resolution Adjustment Network for Existing Detectorscited by 1
  8. 2024SDformerFlow: Spatiotemporal swin spikeformer for event-based optical flow estimationcited by 1
  9. 2024BadScan: An Architectural Backdoor Attack on Visual State Space Modelscited by 1
  10. 2025FractalMamba++: Scaling Vision Mamba Across Resolutions via Hilbert Fractal Geometrycited by 1
  11. 2025Breaking Scale Anchoring: Frequency Representation Learning for Accurate High-Resolution Inference from Low-Resolution Trainingcited by 1
  12. 2026MuViT: Multi-Resolution Vision Transformers for Learning Across Scales in Microscopycited by 1
  13. 2026VibeToken: Scaling 1D Image Tokenizers and Autoregressive Models for Dynamic Resolution Generationscited by 1
  14. 2024Retina Vision Transformer (RetinaViT): Introducing Scaled Patches into Vision Transformerscited by 0
  15. 2025Mitigating Resolution-Drift in Federated Learning: Case of Keypoint Detectioncited by 0
  16. 2025PMRT: A Training Recipe for Fast, 3D High-Resolution Aerodynamic Predictioncited by 0
  17. 2025Task-Aware Resolution Optimization for Visual Large Language Modelscited by 0
  18. 2025On the Use of Hierarchical Vision Foundation Models for Low-Cost Human Mesh Recovery and Pose Estimationcited by 0
  19. 2025StretchySnake: Flexible SSM Training Unlocks Action Recognition Across Spatio-Temporal Scalescited by 0
  20. 2026On What We Can Learn from Low-Resolution Datacited by 0
  21. 2026ProgResViT: Progressive Resolution and Width for Adaptive Vision Transformerscited by 0
  22. 2026FAVE: Foveated Adaptive Visual Encoding for Efficient Fine-Grained Visual Understandingcited by 0