mlsanityData to 7 Oct 2026

Conditional DETR for Fast Training Convergence

arXiv 2108.06152 ↗First posted 13 Aug 2021cs.CVcited by 422

Not on the map. Why?

Abstract (arXiv)

The recently-developed DETR approach applies the transformer encoder and decoder architecture to object detection and achieves promising performance. In this paper, we handle the critical issue, slow training convergence, and present a conditional cross-attention mechanism for fast DETR training. Our approach is motivated by that the cross-attention in DETR relies highly on the content embeddings for localizing the four extremities and predicting the box, which increases the need for high-quality content embeddings and thus the training difficulty. Our approach, named conditional DETR, learns a conditional spatial query from the decoder embedding for decoder multi-head cross-attention. The benefit is that through the conditional spatial query, each cross-attention head is able to attend to a band containing a distinct region, e.g., one object extremity or a region inside the object box. This narrows down the spatial range for localizing the distinct regions for object classification and box regression, thus relaxing the dependence on the content embeddings and easing the training. Empirical results show that conditional DETR converges 6.7x faster for the backbones R50 and R101 and 10x faster for stronger backbones DC5-R50 and DC5-R101. Code is available at https://github.com/Atten4Vis/ConditionalDETR.

Key facts. Cited by 422 papers in the map (data through 7 Oct 2026). Builds on 48 of its 55 references that are in the map. First posted on arXiv 13 Aug 2021.

Builds on

48 of 55 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  7. 2017Squeeze-and-Excitation Networkscited by 4,763
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  9. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  10. 2015Fast R-CNNcited by 2,930
  11. 2018YOLOv3: An Incremental Improvementcited by 2,652
  12. 2017Decoupled Weight Decay Regularizationcited by 1,722
36 more
  1. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  2. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  3. 2016YOLO9000: Better, Faster, Strongercited by 1,429
  4. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  5. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  6. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  7. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  8. 2019Objects as Pointscited by 1,172
  9. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  10. 2017Focal Loss for Dense Object Detectioncited by 749
  11. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  12. 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 539
  13. 2019Dynamic Convolution: Attention over Convolution Kernelscited by 355
  14. 2016Dynamic Filter Networkscited by 318
  15. 2020Conditional Convolutions for Instance Segmentationcited by 314
  16. 2019CondConv: Conditionally Parameterized Convolutions for Efficient Inferencecited by 273
  17. 2019Libra R-CNN: Towards Balanced Learning for Object Detectioncited by 255
  18. 2016UnitBox: An Advanced Object Detection Networkcited by 205
  19. 2020Rethinking Positional Encoding in Language Pre-trainingcited by 197
  20. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  21. 2019Scale-Aware Trident Networks for Object Detectioncited by 159
  22. 2019Bottom-up Object Detection by Grouping Extreme and Center Pointscited by 156
  23. 2020Rethinking Transformer-based Set Prediction for Object Detectioncited by 156
  24. 2019Feature Selective Anchor-Free Module for Single-Shot Object Detectioncited by 139
  25. 2018Gather-Excite: Exploiting Feature Context in Convolutional Neural Networkscited by 138
  26. 2021Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regressioncited by 112
  27. 2020End-to-End Object Detection with Adaptive Clustering Transformercited by 109
  28. 2021Lite-HRNet: A Lightweight High-Resolution Networkcited by 91
  29. 2018Grid R-CNNcited by 85
  30. 2020Revisiting the Sibling Head in Object Detectorcited by 82
  31. 2021Line Segment Detection Using Transformers without Edgescited by 73
  32. 2015DenseBox: Unifying Landmark Localization with End to End Object Detectioncited by 58
  33. 2019Soft Anchor-Point Object Detectioncited by 42
  34. 2019FoveaBox: Beyond Anchor-based Object Detectorcited by 37
  35. 2019T-GSA: Transformer with Gaussian-weighted self-attention for speech enhancementcited by 34
  36. 2019CornerNet-Lite: Efficient Keypoint Based Object Detectioncited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 97; 2023: 112; 2024: 103; 2025: 63; 2026: 41.

422 papers.

Sort:

By area:

  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  2. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  3. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  4. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  5. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  6. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  7. 2023DETRs Beat YOLOs on Real-time Object Detectioncited by 526
  8. 2024YOLOv10: Real-Time End-to-End Object Detectioncited by 508
  9. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 486
  10. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  11. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  12. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
38 more
  1. 2025YOLOv12: Attention-Centric Real-Time Object Detectorscited by 300
  2. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 273
  3. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 267
  4. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  5. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  6. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 192
  7. 2023Query-Dependent Video Representation for Moment Retrieval and Highlight Detectioncited by 166
  8. 2022DETRs with Hybrid Matchingcited by 149
  9. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 149
  10. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  11. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  12. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  13. 2021A Survey of Visual Transformerscited by 117
  14. 2021SGTR: End-to-end Scene Graph Generation with Transformercited by 102
  15. 2022CenterFormer: Center-based Transformer for 3D Object Detectioncited by 99
  16. 2022Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignmentcited by 96
  17. 2022Learning Object-Language Alignments for Open-Vocabulary Object Detectioncited by 95
  18. 2022MonoDETR: Depth-guided Transformer for Monocular 3D Object Detectioncited by 85
  19. 2023Knowing Where to Focus: Event-aware Transformer for Video Groundingcited by 79
  20. 2022AdaMixer: A Fast-Converging Query-Based Object Detectorcited by 78
  21. 2023End-to-End 3D Dense Captioning with Vote2Cap-DETRcited by 77
  22. 2022An End-to-End Transformer Model for Crowd Localizationcited by 74
  23. 2024D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinementcited by 73
  24. 2025RF-DETR: Neural Architecture Search for Real-Time Detection Transformerscited by 70
  25. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 69
  26. 2024DEIM: DETR with Improved Matching for Fast Convergencecited by 67
  27. 2023SPTS v2: Single-Point Scene Text Spottingcited by 66
  28. 2023SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applicationscited by 66
  29. 2022Few Could Be Better Than All: Feature Sampling and Grouping for Scene Text Detectioncited by 65
  30. 2022Accelerating DETR Convergence via Semantic-Aligned Matchingcited by 58
  31. 2023Exploring Predicate Visual Context in Detecting Human-Object Interactionscited by 56
  32. 2024OPUS: Occupancy Prediction Using a Sparse Setcited by 56
  33. 2022Temporally Efficient Vision Transformer for Video Instance Segmentationcited by 55
  34. 2022AO2-DETR: Arbitrary-Oriented Object Detection Transformercited by 55
  35. 2024SHViT: Single-Head Vision Transformer with Memory Efficient Macro Designcited by 55
  36. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
  37. 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
  38. 2022DPText-DETR: Towards Better Scene Text Detection with Dynamic Points in Transformercited by 50

50 most cited of 422 All 422 (JSON)