mlsanityData to 7 Oct 2026

Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions

arXiv 2102.12122 ↗First posted 24 Feb 2021cs.CVcited by 1,960

Not on the map. Why?

Cited by 43 in State-space sequence models (S4, Mamba), but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Although using convolutional neural networks (CNNs) as backbones achieves great successes in computer vision, this work investigates a simple backbone network useful for many dense prediction tasks without convolutions. Unlike the recently-proposed Transformer model (e.g., ViT) that is specially designed for image classification, we propose Pyramid Vision Transformer~(PVT), which overcomes the difficulties of porting Transformer to various dense prediction tasks. PVT has several merits compared to prior arts. (1) Different from ViT that typically has low-resolution outputs and high computational and memory cost, PVT can be not only trained on dense partitions of the image to achieve high output resolution, which is important for dense predictions but also using a progressive shrinking pyramid to reduce computations of large feature maps. (2) PVT inherits the advantages from both CNN and Transformer, making it a unified backbone in various vision tasks without convolutions by simply replacing CNN backbones. (3) We validate PVT by conducting extensive experiments, showing that it boosts the performance of many downstream tasks, e.g., object detection, semantic, and instance segmentation. For example, with a comparable number of parameters, RetinaNet+PVT achieves 40.4 AP on the COCO dataset, surpassing RetinNet+ResNet50 (36.3 AP) by 4.1 absolute AP. We hope PVT could serve as an alternative and useful backbone for pixel-level predictions and facilitate future researches. Code is available at https://github.com/whai362/PVT.

Key facts. Cited by 1,960 papers in the map (data through 7 Oct 2026). Builds on 69 of its 89 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

69 of 89 references.

References · 69

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  9. 2020End-to-End Object Detection with Transformerscited by 7,384
  10. 2017Mask R-CNNcited by 7,259
  11. 2016Densely Connected Convolutional Networkscited by 6,691
  12. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
38 more
  1. 2016Layer Normalizationcited by 5,579
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2017Squeeze-and-Excitation Networkscited by 4,763
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2014Going Deeper with Convolutionscited by 4,231
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  11. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  12. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  13. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  14. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  15. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  16. 2016Pyramid Scene Parsing Networkcited by 2,472
  17. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  18. 2017Non-local Neural Networkscited by 2,290
  19. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  20. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  21. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  22. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  23. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  24. 2017Random Erasing Data Augmentationcited by 1,235
  25. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  26. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  27. 2017Focal Loss for Dense Object Detectioncited by 749
  28. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  29. 2019Panoptic Feature Pyramid Networkscited by 644
  30. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  31. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 494
  32. 2019Res2Net: A New Multi-scale Backbone Architecturecited by 468
  33. 2020PraNet: Parallel Reverse Attention Network for Polyp Segmentationcited by 460
  34. 2020ResNeSt: Split-Attention Networkscited by 445
  35. 2020Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detectioncited by 408
  36. 2019Selective Kernel Networkscited by 401
  37. 2020TransTrack: Multiple Object Tracking with Transformercited by 370
  38. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 227; 2022: 490; 2023: 502; 2024: 380; 2025: 236; 2026: 125.

1,960 papers.

Sort:

By area:

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  2. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  3. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  4. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  5. 2021ViViT: A Video Vision Transformercited by 1,610
  6. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  7. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
  8. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  9. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  10. 2021Scaling Vision Transformerscited by 993
  11. 2024VMamba: Visual State Space Modelcited by 955
  12. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 954
38 more
  1. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  2. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  3. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
  4. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  5. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  6. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 812
  7. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  8. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  9. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  10. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  11. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 636
  12. 2021Transformer in Transformercited by 632
  13. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  14. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  15. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  16. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  17. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 530
  18. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  19. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  20. 2021MetaFormer Is Actually What You Need for Visioncited by 523
  21. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516
  22. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  23. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 479
  24. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  25. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  26. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  27. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 450
  28. 2022Activating More Pixels in Image Super-Resolution Transformercited by 433
  29. 2024YOLOv9: Learning What You Want to Learn Using Programmable Gradient Informationcited by 422
  30. 2022DeiT III: Revenge of the ViTcited by 417
  31. 2021Early Convolutions Help Transformers See Bettercited by 395
  32. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 384
  33. 2021Pay Attention to MLPscited by 367
  34. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
  35. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 359
  36. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 357
  37. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  38. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 326

50 most cited of 1,960 All 1,960 (JSON)