mlsanityData to 7 Oct 2026

MViTv2: Improved Multiscale Vision Transformers for Classification and Detection

arXiv 2112.01526 ↗First posted 2 Dec 2021cs.CVcited by 516

Not on the map. Why?

Abstract (arXiv)

In this paper, we study Multiscale Vision Transformers (MViTv2) as a unified architecture for image and video classification, as well as object detection. We present an improved version of MViT that incorporates decomposed relative positional embeddings and residual pooling connections. We instantiate this architecture in five sizes and evaluate it for ImageNet classification, COCO detection and Kinetics video recognition where it outperforms prior work. We further compare MViTv2s' pooling attention to window attention mechanisms where it outperforms the latter in accuracy/compute. Without bells-and-whistles, MViTv2 has state-of-the-art performance in 3 domains: 88.8% accuracy on ImageNet classification, 58.7 boxAP on COCO object detection as well as 86.1% on Kinetics-400 video classification. Code and models are available at https://github.com/facebookresearch/mvit.

Key facts. Cited by 516 papers in the map (data through 7 Oct 2026). Builds on 82 of its 90 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

82 of 90 references.

References · 82

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  8. 2017Mask R-CNNcited by 7,259
  9. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  10. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  12. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
38 more
  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  2. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2014Going Deeper with Convolutionscited by 4,231
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  7. 2015Fast R-CNNcited by 2,930
  8. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  9. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  12. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  13. 2018SlowFast Networks for Video Recognitioncited by 1,862
  14. 2017Decoupled Weight Decay Regularizationcited by 1,722
  15. 2021ViViT: A Video Vision Transformercited by 1,610
  16. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  17. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  18. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  19. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  20. 2017Random Erasing Data Augmentationcited by 1,235
  21. 2019Objects as Pointscited by 1,172
  22. 2018Self-Attention with Relative Position Representationscited by 1,112
  23. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  24. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  25. 2021Video Swin Transformercited by 1,090
  26. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  27. 2016Deep Networks with Stochastic Depthcited by 973
  28. 2020Designing Network Design Spacescited by 954
  29. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  30. 2021Multiscale Vision Transformerscited by 896
  31. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  32. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  33. 2021Going deeper with Image Transformerscited by 759
  34. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  35. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  36. 2021Transformer in Transformercited by 632
  37. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  38. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 86; 2023: 154; 2024: 127; 2025: 89; 2026: 59.

516 papers.

Sort:

By area:

  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  3. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  4. 2022Token Merging: Your ViT But Fastercited by 995
  5. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  6. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  7. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  8. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  9. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  10. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  11. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  12. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 450
38 more
  1. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  2. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  3. 2023EVA-02: A Visual Representation for Neon Genesiscited by 377
  4. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  5. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 320
  6. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 287
  7. 2022Masked Autoencoders that Listencited by 271
  8. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 240
  9. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  10. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 226
  11. 2023Unmasked Teacher: Towards Training-Efficient Video Foundation Modelscited by 199
  12. 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 192
  13. 2022MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognitioncited by 189
  14. 2022Frozen CLIP Models are Efficient Video Learnerscited by 181
  15. 2022DaViT: Dual Attention Vision Transformerscited by 174
  16. 2022DETRs with Hybrid Matchingcited by 149
  17. 2023AIM: Adapting Image Models for Efficient Video Action Recognitioncited by 145
  18. 2024MambaOut: Do We Really Need Mamba for Vision?cited by 142
  19. 2024PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognitioncited by 141
  20. 2022MetaFormer Baselines for Visioncited by 131
  21. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  22. 2023RepViT: Revisiting Mobile CNN From ViT Perspectivecited by 123
  23. 2022EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applicationscited by 121
  24. 2022Rethinking Vision Transformers for MobileNet Size and Speedcited by 121
  25. 2022UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormercited by 116
  26. 2022ConvMAE: Masked Convolution Meets Masked Autoencoderscited by 105
  27. 2022HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutionscited by 103
  28. 2024Demystify Mamba in Vision: A Linear Attention Perspectivecited by 101
  29. 2022OmniMAE: Single Model Masked Pretraining on Images and Videoscited by 99
  30. 20223D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentationcited by 94
  31. 2022Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learningcited by 90
  32. 2023Vita-CLIP: Video and text adaptive CLIP via Multimodal Promptingcited by 90
  33. 2023Agent Attention: On the Integration of Softmax and Linear Attentioncited by 90
  34. 2023InceptionNeXt: When Inception Meets ConvNeXtcited by 84
  35. 2024Mamba-ND: Selective State Space Modeling for Multi-Dimensional Datacited by 84
  36. 2022Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognitioncited by 81
  37. 2022HiTeA: Hierarchical Temporal-Aware Video-Language Pre-trainingcited by 72
  38. 2023Diffusion Models as Masked Autoencoderscited by 71

50 most cited of 516 All 516 (JSON)