mlsanityData to 7 Oct 2026

DETRs Beat YOLOs on Real-time Object Detection

arXiv 2304.08069 ↗First posted 17 Apr 2023cs.CVcited by 526

Not on the map. Why?

Abstract (arXiv)

The YOLO series has become the most popular framework for real-time object detection due to its reasonable trade-off between speed and accuracy. However, we observe that the speed and accuracy of YOLOs are negatively affected by the NMS. Recently, end-to-end Transformer-based detectors (DETRs) have provided an alternative to eliminating NMS. Nevertheless, the high computational cost limits their practicality and hinders them from fully exploiting the advantage of excluding NMS. In this paper, we propose the Real-Time DEtection TRansformer (RT-DETR), the first real-time end-to-end object detector to our best knowledge that addresses the above dilemma. We build RT-DETR in two steps, drawing on the advanced DETR: first we focus on maintaining accuracy while improving speed, followed by maintaining speed while improving accuracy. Specifically, we design an efficient hybrid encoder to expeditiously process multi-scale features by decoupling intra-scale interaction and cross-scale fusion to improve speed. Then, we propose the uncertainty-minimal query selection to provide high-quality initial queries to the decoder, thereby improving accuracy. In addition, RT-DETR supports flexible speed tuning by adjusting the number of decoder layers to adapt to various scenarios without retraining. Our RT-DETR-R50 / R101 achieves 53.1% / 54.3% AP on COCO and 108 / 74 FPS on T4 GPU, outperforming previously advanced YOLOs in both speed and accuracy. We also develop scaled RT-DETRs that outperform the lighter YOLO detectors (S and M models). Furthermore, RT-DETR-R50 outperforms DINO-R50 by 2.2% AP in accuracy and about 21 times in FPS. After pre-training with Objects365, RT-DETR-R50 / R101 achieves 55.3% / 56.2% AP. The project page: https://zhao-yian.github.io/RTDETR.

Key facts. Cited by 526 papers in the map (data through 7 Oct 2026). Builds on 48 of its 61 references that are in the map. First posted on arXiv 17 Apr 2023.

Builds on

48 of 61 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  9. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  10. 2015Fast R-CNNcited by 2,930
  11. 2018YOLOv3: An Incremental Improvementcited by 2,652
  12. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
36 more
  1. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  2. 2016YOLO9000: Better, Faster, Strongercited by 1,429
  3. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  4. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  5. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  6. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  7. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  8. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 812
  9. 2018Path Aggregation Network for Instance Segmentationcited by 796
  10. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  11. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  12. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 494
  13. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 486
  14. 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 430
  15. 2021Conditional DETR for Fast Training Convergencecited by 422
  16. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
  17. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
  18. 2019NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detectioncited by 289
  19. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 273
  20. 2020VarifocalNet: An IoU-aware Dense Object Detectorcited by 230
  21. 2020Scaled-YOLOv4: Scaling Cross Stage Partial Networkcited by 211
  22. 2022Focal Modulation Networkscited by 186
  23. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  24. 2021Efficient DETR: Improving End-to-End Object Detector with Dense Priorcited by 128
  25. 2021Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsitycited by 98
  26. 2022Anomaly Detection in Autonomous Driving: A Surveycited by 87
  27. 2022PP-YOLOE: An evolved version of YOLOcited by 75
  28. 2020End-to-End Object Detection with Fully Convolutional Networkcited by 70
  29. 2023YOLOv6 v3.0: A Full-Scale Reloadingcited by 58
  30. 2020PP-YOLO: An Effective and Efficient Implementation of Object Detectorcited by 44
  31. 2023Lite DETR : An Interleaved Multi-Scale Encoder for Efficient DETRcited by 36
  32. 2022Reversible Column Networkscited by 32
  33. 2021PP-YOLOv2: A Practical Object Detectorcited by 30
  34. 2022Group DETR v2: Strong Object Detector with Encoder-Decoder Pretrainingcited by 25
  35. 2021Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbonescited by 15
  36. 2022D^2ETR: Decoder-Only DETR with Computationally Efficient Cross-Scale Attentioncited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 20; 2024: 116; 2025: 203; 2026: 187.

526 papers.

Sort:

By area:

  1. 2024YOLOv10: Real-Time End-to-End Object Detectioncited by 508
  2. 2024YOLOv9: Learning What You Want to Learn Using Programmable Gradient Informationcited by 422
  3. 2025YOLOv12: Attention-Centric Real-Time Object Detectorscited by 300
  4. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 224
  5. 2024Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detectioncited by 122
  6. 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 81
  7. 2024D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinementcited by 73
  8. 2025RF-DETR: Neural Architecture Search for Real-Time Detection Transformerscited by 70
  9. 2024DEIM: DETR with Improved Matching for Fast Convergencecited by 67
  10. 2025Detect Anything via Next Point Predictioncited by 66
  11. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 59
  12. 2024Multiple Object Tracking as ID Predictioncited by 58
38 more
  1. 2023Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Reviewcited by 57
  2. 2025YOLOv13: Real-Time Object Detection with Hypergraph-Enhanced Adaptive Visual Perceptioncited by 57
  3. 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 54
  4. 2023Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanismcited by 51
  5. 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
  6. 2026Ultralytics YOLO26: Unified Real-Time End-to-End Vision Modelscited by 47
  7. 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Modelcited by 44
  8. 2023Change Detection Methods for Remote Sensing in the Last Decade: A Comprehensive Reviewcited by 41
  9. 2024LW-DETR: A Transformer Replacement to YOLO for Real-Time Detectioncited by 36
  10. 2023YOLO-MS: Rethinking Multi-Scale Representation Learning for Real-time Object Detectioncited by 33
  11. 2023RTMO: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimationcited by 33
  12. 2024RT-DETRv3: Real-time End-to-End Object Detection with Hierarchical Dense Positive Supervisioncited by 31
  13. 2024Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentationcited by 30
  14. 2024Docling Technical Reportcited by 29
  15. 2024DAMSDet: Dynamic Adaptive Multispectral Detection Transformer with Competitive Query Selection and Adaptive Feature Fusioncited by 28
  16. 2025PP-DocLayout: A Unified Document Layout Detection Model to Accelerate Large-Scale Data Constructioncited by 26
  17. 2025Real-Time Object Detection Meets DINOv3cited by 26
  18. 2024HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interactioncited by 25
  19. 2024Object Detection using Event Camera: A MoE Heat Conduction based Detector and A New Benchmark Datasetcited by 25
  20. 2024HumanRefiner: Benchmarking Abnormal Human Generation and Refining with Coarse-to-fine Pose-Reversible Guidancecited by 24
  21. 2024Dataset Enhancement with Instance-Level Augmentationscited by 23
  22. 2023Object Detection with Transformers: A Reviewcited by 22
  23. 2025FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interactioncited by 22
  24. 2024Real-time Transformer-based Open-Vocabulary Detection with Efficient Fusion Headcited by 19
  25. 2025Building a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMscited by 19
  26. 2025VASparse: Towards Efficient Visual Hallucination Mitigation via Visual-Aware Token Sparsificationcited by 17
  27. 2025FRED: The Florence RGB-Event Drone Datasetcited by 16
  28. 2024AutoSoccerPose: Automated 3D posture Analysis of Soccer Shot Movementscited by 15
  29. 2024HazyDet: Open-Source Benchmark for Drone-View Object Detection with Depth-Cues in Hazy Scenescited by 15
  30. 2025UAV-DETR: Efficient End-to-End Object Detection for Unmanned Aerial Vehicle Imagerycited by 15
  31. 2024Pytorch-Wildlife: A Collaborative Deep Learning Framework for Conservationcited by 14
  32. 2025Object Detection with Multimodal Large Vision-Language Models: An In-depth Reviewcited by 14
  33. 2024GraCo: Granularity-Controllable Interactive Segmentationcited by 13
  34. 2025MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patternscited by 13
  35. 2025VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detectioncited by 12
  36. 2024SGSeg: Enabling Text-free Inference in Language-guided Segmentation of Chest X-rays via Self-guidancecited by 11
  37. 2024Online Episodic Memory Visual Query Localization with Egocentric Streaming Object Memorycited by 11
  38. 2025RF-DETR Object Detection vs YOLOv12 : A Study of Transformer-based and CNN-based Architectures for Single-Class and Multi-Class Greenfruit Detection in Complex Orchard Environments Under Label Ambiguitycited by 11

50 most cited of 526 All 526 (JSON)