mlsanityData to 7 Oct 2026

End-to-End Object Detection with Adaptive Clustering Transformer

arXiv 2011.09315 ↗First posted 18 Nov 2020cs.CVcited by 109

Not on the map. Why?

Abstract (arXiv)

End-to-end Object Detection with Transformer (DETR)proposes to perform object detection with Transformer and achieve comparable performance with two-stage object detection like Faster-RCNN. However, DETR needs huge computational resources for training and inference due to the high-resolution spatial input. In this paper, a novel variant of transformer named Adaptive Clustering Transformer(ACT) has been proposed to reduce the computation cost for high-resolution input. ACT cluster the query features adaptively using Locality Sensitive Hashing (LSH) and ap-proximate the query-key interaction using the prototype-key interaction. ACT can reduce the quadratic O(N2) complexity inside self-attention into O(NK) where K is the number of prototypes in each layer. ACT can be a drop-in module replacing the original self-attention module without any training. ACT achieves a good balance between accuracy and computation cost (FLOPs). The code is available as supplementary for the ease of experiment replication and verification. Code is released at urlhttps://github.com/gaopengcuhk/SMCA-DETR/

Key facts. Cited by 109 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 28; 2022: 33; 2023: 26; 2024: 14; 2025: 8.

109 papers.

Sort:

By area:

  1. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  2. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  3. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  4. 2021Conditional DETR for Fast Training Convergencecited by 422
  5. 2021A Survey of Transformerscited by 399
  6. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 355
  7. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 286
  8. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  9. 2023PixelLM: Pixel Reasoning with Large Multimodal Modelcited by 220
  10. 2021Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encodingcited by 207
  11. 2022Focal Modulation Networkscited by 186
  12. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
38 more
  1. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  2. 2021Instances as Queriescited by 157
  3. 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
  4. 2021All Tokens Matter: Token Labeling for Training Better Vision Transformerscited by 144
  5. 2022Understanding The Robustness in Vision Transformerscited by 144
  6. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 139
  7. 2022Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practicecited by 136
  8. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  9. 2021A Survey of Visual Transformerscited by 117
  10. 2023Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policycited by 104
  11. 2022CALIP: Zero-Shot Enhancement of CLIP with Parameter-free Attentioncited by 103
  12. 2021FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpaintingcited by 94
  13. 2022MonoDETR: Depth-guided Transformer for Monocular 3D Object Detectioncited by 85
  14. 2021OadTR: Online Action Detection with Transformerscited by 83
  15. 2022Unified Visual Transformer Compressioncited by 79
  16. 2022Boosting Crowd Counting via Multifaceted Attentioncited by 73
  17. 2023Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformerscited by 59
  18. 2023Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentationcited by 54
  19. 2023PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detectioncited by 53
  20. 2021PnP-DETR: Towards Efficient Visual Analysis with Transformerscited by 52
  21. 2021Vision Transformer with Progressive Samplingcited by 44
  22. 2023HiCLIP: Contrastive Language-Image Pretraining with Hierarchy-aware Attentioncited by 44
  23. 2022Vision Transformer with Super Token Samplingcited by 43
  24. 2023The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Mattercited by 42
  25. 2022Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Groundingcited by 40
  26. 2021Refiner: Refining Self-attention for Vision Transformerscited by 38
  27. 2021Container: Context Aggregation Networkcited by 37
  28. 2022The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancycited by 35
  29. 2022Self-Supervised Masked Convolutional Transformer Block for Anomaly Detectioncited by 34
  30. 2022BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-trainingcited by 33
  31. 2022Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuningcited by 33
  32. 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 32
  33. 2023OcTr: Octree-based Transformer for 3D Object Detectioncited by 32
  34. 2021Light Field Image Super-Resolution with Transformerscited by 31
  35. 2022Progressive End-to-End Object Detection in Crowded Scenescited by 31
  36. 2022iQuery: Instruments as Queries for Audio-Visual Sound Separationcited by 29
  37. 2024Hyper-YOLO: When Visual Object Detection Meets Hypergraph Computationcited by 29
  38. 2023Making Vision Transformers Efficient from A Token Sparsification Viewcited by 28

50 most cited of 109 All 109 (JSON)