mlsanityData to 7 Oct 2026

DPT: Deformable Patch-based Transformer for Visual Recognition

arXiv 2107.14467 ↗First posted 30 Jul 2021cs.CVcited by 43

Not on the map. Why?

Abstract (arXiv)

Transformer has achieved great success in computer vision, while how to split patches in an image remains a problem. Existing methods usually use a fixed-size patch embedding which might destroy the semantics of objects. To address this problem, we propose a new Deformable Patch (DePatch) module which learns to adaptively split the images into patches with different positions and scales in a data-driven way rather than using predefined fixed patches. In this way, our method can well preserve the semantics in patches. The DePatch module can work as a plug-and-play module, which can easily be incorporated into different transformers to achieve an end-to-end training. We term this DePatch-embedded transformer as Deformable Patch-based Transformer (DPT) and conduct extensive evaluations of DPT on image classification and object detection. Results show DPT can achieve 81.9% top-1 accuracy on ImageNet classification, and 43.7% box mAP with RetinaNet, 44.3% with Mask R-CNN on MSCOCO object detection. Code has been made available at: https://github.com/CASIA-IVA-Lab/DPT .

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 31 of its 37 references that are in the map. First posted on arXiv 30 Jul 2021.

Builds on

31 of 37 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2020End-to-End Object Detection with Transformerscited by 7,384
  9. 2017Mask R-CNNcited by 7,259
  10. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  12. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
19 more
  1. 2017Squeeze-and-Excitation Networkscited by 4,763
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  5. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  6. 2017Non-local Neural Networkscited by 2,290
  7. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  9. 2017Deformable Convolutional Networkscited by 1,481
  10. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  11. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  12. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  13. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 666
  14. 2021Transformer in Transformercited by 632
  15. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  16. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  17. 2019GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyondcited by 331
  18. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  19. 2020Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classificationcited by 99

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 14; 2023: 13; 2024: 7; 2025: 5; 2026: 2.

43 papers.

Sort:

By area:

  1. 2022Vision Transformer with Deformable Attentioncited by 293
  2. 2023BiFormer: Vision Transformer with Bi-Level Routing Attentioncited by 150
  3. 2022Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentationcited by 71
  4. 2022Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentationcited by 46
  5. 2023Vision Transformer with Quadrangle Attentioncited by 46
  6. 2022Towards Practical Certifiable Patch Defense with Vision Transformercited by 44
  7. 2022VSA: Learning Varied-Size Window Attention in Vision Transformerscited by 43
  8. 2021Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attentioncited by 36
  9. 2023A survey of the Vision Transformers and their CNN-Transformer based Variantscited by 33
  10. 2022UniVIP: A Unified Framework for Self-Supervised Visual Pre-trainingcited by 30
  11. 2022Learning Graph Neural Networks for Image Style Transfercited by 29
  12. 2022EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithmcited by 27
31 more
  1. 2024ConvTimeNet: A Deep Hierarchical Fully Convolutional Model for Multivariate Time Series Analysiscited by 26
  2. 2023SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentationcited by 25
  3. 2023Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generationcited by 25
  4. 2023A Recent Survey of Vision Transformers for Medical Image Segmentationcited by 22
  5. 2023DAT++: Spatially Dynamic Vision Transformer with Deformable Attentioncited by 21
  6. 2023GTP-ViT: Efficient Vision Transformers via Graph-based Token Propagationcited by 19
  7. 2021ELSA: Enhanced Local Self-Attention for Vision Transformercited by 18
  8. 2024PixelGaussian: Generalizable 3D Gaussian Reconstruction from Arbitrary Viewscited by 18
  9. 2025DefMamba: Deformable Visual State Space Modelcited by 18
  10. 2022Obj2Seq: Formatting Objects as Sequences with Class Prompt for Visual Taskscited by 12
  11. 2024A Review of Transformer-Based Models for Computer Vision Tasks: Capturing Global Context and Spatial Relationshipscited by 11
  12. 2022What Makes for Good Tokenizers in Vision Transformer?cited by 9
  13. 2023Stable Segment Anything Modelcited by 6
  14. 2024DeformTime: Capturing Variable Dependencies with Deformable Attention for Time Series Forecastingcited by 6
  15. 2022Learning to Incorporate Texture Saliency Adaptive Attention to Image Cartoonizationcited by 5
  16. 2023Student Classroom Behavior Detection based on YOLOv7-BRA and Multi-Model Fusioncited by 5
  17. 2024DeBiFormer: Vision Transformer with Deformable Agent Bi-level Routing Attentioncited by 4
  18. 2025DriveVGGT: Calibration-Constrained Visual Geometry Transformers for Multi-Camera Autonomous Drivingcited by 4
  19. 2023Student Classroom Behavior Detection based on Improved YOLOv7cited by 3
  20. 2023Vision Backbone Enhancement via Multi-Stage Cross-Scale Attentioncited by 3
  21. 2025Differentiable Hierarchical Visual Tokenizationcited by 3
  22. 2022Improving Semantic Segmentation in Transformers using Hierarchical Inter-Level Attentioncited by 2
  23. 2022ROIFormer: Semantic-Aware Region of Interest Transformer for Efficient Self-Supervised Monocular Depth Estimationcited by 2
  24. 2024SCLNet: A Scale-Robust Complementary Learning Network for Object Detection in UAV Imagescited by 1
  25. 2025Adaptive Morph-Patch Transformer for Aortic Vessel Segmentationcited by 1
  26. 2026Mixture-of-Top-k Attention: Efficient Attention via Scalable Fast Weightscited by 1
  27. 2026Hierarchical Molecular Representation Learning via Fragment-Based Self-Supervised Embedding Predictioncited by 1
  28. 2022PicT: A Slim Weakly Supervised Vision Transformer for Pavement Distress Classificationcited by 0
  29. 2023Deformable Audio Transformer for Audio Event Detectioncited by 0
  30. 2024Understanding the Effect of using Semantically Meaningful Tokens for Visual Representation Learningcited by 0
  31. 2025Airport Passenger Flow Forecasting via Deformable Temporal-Spectral Transformer Approachcited by 0