mlsanityData to 7 Oct 2026

ResT V2: Simpler, Faster and Stronger

arXiv 2204.07366 ↗First posted 15 Apr 2022cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper proposes ResTv2, a simpler, faster, and stronger multi-scale vision Transformer for visual recognition. ResTv2 simplifies the EMSA structure in ResTv1 (i.e., eliminating the multi-head interaction part) and employs an upsample operation to reconstruct the lost medium- and high-frequency information caused by the downsampling operation. In addition, we explore different techniques for better apply ResTv2 backbones to downstream tasks. We found that although combining EMSAv2 and window attention can greatly reduce the theoretical matrix multiply FLOPs, it may significantly decrease the computation density, thus causing lower actual speed. We comprehensively validate ResTv2 on ImageNet classification, COCO detection, and ADE20K semantic segmentation. Experimental results show that the proposed ResTv2 can outperform the recently state-of-the-art backbones by a large margin, demonstrating the potential of ResTv2 as solid backbones. The code and models will be made publicly available at urlhttps://github.com/wofmanaf/ResT

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 15 Apr 2022.

Builds on

45 of 64 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2017Mask R-CNNcited by 7,259
  8. 2016Layer Normalizationcited by 5,579
  9. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  10. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  11. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  12. 2022A ConvNet for the 2020scited by 3,545
33 more
  1. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  2. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  3. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  4. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  5. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  6. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  7. 2017Random Erasing Data Augmentationcited by 1,235
  8. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  9. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  10. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  11. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  12. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  13. 2021Scaling Vision Transformerscited by 993
  14. 2016Deep Networks with Stochastic Depthcited by 973
  15. 2020Designing Network Design Spacescited by 954
  16. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  17. 2021Multiscale Vision Transformerscited by 896
  18. 2020Pre-Trained Image Processing Transformercited by 891
  19. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  20. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  21. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  22. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  23. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  24. 2021Early Convolutions Help Transformers See Bettercited by 395
  25. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  26. 2022How Do Vision Transformers Work?cited by 305
  27. 2021Mobile-Former: Bridging MobileNet and Transformercited by 234
  28. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 182
  29. 2021Conformer: Local Features Coupling Global Representations for Visual Recognitioncited by 164
  30. 2021Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformercited by 115
  31. 2021ResT: An Efficient Transformer for Visual Recognitioncited by 79
  32. 2021On the Integration of Self-Attention and Convolutioncited by 74
  33. 2021Glance-and-Gaze Vision Transformercited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 1; 2026: 1.

9 papers.

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2023SG-Former: Self-guided Transformer with Evolving Token Reallocationcited by 32
  3. 2024SMAFormer: Synergistic Multi-Attention Transformer for Medical Image Segmentationcited by 14
  4. 2024Towards a Comprehensive, Efficient and Promptable Anatomic Structure Segmentation Model using 3D Whole-body CT Scanscited by 10
  5. 2024Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding Samplecited by 6
  6. 2024Lagrange Duality and Compound Multi-Attention Transformer for Semi-Supervised Medical Image Segmentationcited by 2
  7. 2023EViT: An Eagle Vision Transformer with Bi-Fovea Self-Attentioncited by 1
  8. 2025HBFormer: A Hybrid-Bridge Transformer for Microtumor and Miniature Organ Segmentationcited by 0
  9. 2026Insights from Visual Cognition: Understanding Human Action Dynamics with Overall Glance and Refined Gaze Transformercited by 0