# RT-DETRv3: Real-time End-to-End Object Detection with Hierarchical Dense Positive Supervision

arXiv: https://arxiv.org/abs/2409.08475 · HTML: https://mlsanity.com/paper/2409.08475/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 32 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [DETRs Beat YOLOs on Real-time Object Detection](https://mlsanity.com/paper/2304.08069/) (2023)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [YOLOv10: Real-Time End-to-End Object Detection](https://mlsanity.com/paper/2405.14458/) (2024)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [CSPNet: A New Backbone that can Enhance Learning Capability of CNN](https://mlsanity.com/paper/1911.11929/) (2019)
- [YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications](https://mlsanity.com/paper/2209.02976/) (2022)
- [DETRs with Collaborative Hybrid Assignments Training](https://mlsanity.com/paper/2211.12860/) (2022)
- [Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignment](https://mlsanity.com/paper/2207.13085/) (2022)
- [RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer](https://mlsanity.com/paper/2407.17140/) (2024)
- [PP-YOLOE: An evolved version of YOLO](https://mlsanity.com/paper/2203.16250/) (2022)
- [ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions](https://mlsanity.com/paper/2403.07392/) (2024)
- [Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism](https://mlsanity.com/paper/2309.11331/) (2023)
- [PP-YOLO: An Effective and Efficient Implementation of Object Detector](https://mlsanity.com/paper/2007.12099/) (2020)
- [MS-DETR: Efficient DETR Training with Mixed Supervision](https://mlsanity.com/paper/2401.03989/) (2024)
- [YOLO-MS: Rethinking Multi-Scale Representation Learning for Real-time Object Detection](https://mlsanity.com/paper/2308.05480/) (2023)
- [PP-YOLOv2: A Practical Object Detector](https://mlsanity.com/paper/2104.10419/) (2021)

## Built on by (most cited first, 30 of 32)

- [YOLOv12: Attention-Centric Real-Time Object Detectors](https://mlsanity.com/paper/2502.12524/) (2025)
- [YOLO26: Key Architectural Enhancements and Performance Benchmarking for Real-Time Object Detection](https://mlsanity.com/paper/2509.25164/) (2025)
- [Sat2RealCity: Geometry-Aware and Appearance-Controllable 3D Urban Generation from Satellite Imagery](https://mlsanity.com/paper/2511.11470/) (2025)
- [You Only Gaussian Once: Controllable 3D Gaussian Splatting for Ultra-Densely Sampled Scenes](https://mlsanity.com/paper/2604.21400/) (2026)
- [Enhancing LLM-based Autonomous Driving with Modular Traffic Light and Sign Recognition](https://mlsanity.com/paper/2511.14391/) (2025)
- [EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation](https://mlsanity.com/paper/2603.18739/) (2026)
- [MorphXAI: An Explainable Framework for Morphological Analysis of Parasites in Blood Smear Images](https://mlsanity.com/paper/2601.18001/) (2026)
- [YOLO26: A Comprehensive Architecture Overview and Key Improvements](https://mlsanity.com/paper/2602.14582/) (2026)
- [Real-Time Oriented Object Detection Transformer in Remote Sensing Images](https://mlsanity.com/paper/2603.15497/) (2026)
- [LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detection](https://mlsanity.com/paper/2512.05663/) (2025)
- [CORDS: Continuous Representations of Discrete Structures](https://mlsanity.com/paper/2601.21583/) (2026)
- [Learning on the Fly: Replay-Based Continual Object Perception for Indoor Drones](https://mlsanity.com/paper/2602.13440/) (2026)
- [Le-DETR: Revisiting Real-Time Detection Transformer with Efficient Encoder Design](https://mlsanity.com/paper/2602.21010/) (2026)
- [RiO-DETR: DETR for Real-time Oriented Object Detection](https://mlsanity.com/paper/2603.09411/) (2026)
- [Resource-Constrained UAV-Based Weed Detection for Site-Specific Management on Edge Devices](https://mlsanity.com/paper/2604.23442/) (2026)
- [RT-SDGOD: Real-Time Single-Domain Generalized Object Detection](https://mlsanity.com/paper/2606.09367/) (2026)
- [CP-WSP: A Declarative CP-SAT Framework for Configurable Multi-Constraint Workforce Scheduling](https://mlsanity.com/paper/2607.05177/) (2026)
- [RT-DATR: Real-time Unsupervised Domain Adaptive Detection Transformer with Adversarial Feature Alignment](https://mlsanity.com/paper/2504.09196/) (2025)
- [Hierarchical Neural Collapse Detection Transformer for Class Incremental Object Detection](https://mlsanity.com/paper/2506.08562/) (2025)
- [2.5D Object Detection for Intelligent Roadside Infrastructure](https://mlsanity.com/paper/2507.03564/) (2025)
- [Heatmap Guided Query Transformers for Robust Astrocyte Detection across Immunostains and Resolutions](https://mlsanity.com/paper/2509.03323/) (2025)
- [CorVS+: Correspondence-Driven Association of Video Trajectories and Sensors for Identity-Aware Person Localization in Warehouses](https://mlsanity.com/paper/2510.26369/) (2025)
- [All-Optical Segmentation via Diffractive Neural Networks for Autonomous Driving](https://mlsanity.com/paper/2602.07717/) (2026)
- [Detection of On-Ground Chestnuts Using Artificial Intelligence Toward Automated Picking](https://mlsanity.com/paper/2602.14140/) (2026)
- [Benchmarking ResNet Backbones in RT-DETR: Impact of Depth and Regularization under environmental conditions](https://mlsanity.com/paper/2605.08136/) (2026)
- [DANCE: Detect and Classify Events in EEG](https://mlsanity.com/paper/2605.10688/) (2026)
- [Context-Guided Semantic Alignment for Feature Fusion Networks](https://mlsanity.com/paper/2606.14005/) (2026)
- [Real-Time Source-Free Object Detection](https://mlsanity.com/paper/2606.31834/) (2026)
- [STAR: A Spatial-Topology Aware Routing Framework for Generalizable 3D Scene Understanding](https://mlsanity.com/paper/2608.11699/) (2026)
- [Reading Decoder Trajectories: Training-Free Counterfactual Query-Trajectory Reliability for Small-Object Detection](https://mlsanity.com/paper/2609.06581/) (2026)

## Abstract (arXiv)

RT-DETR is the first real-time end-to-end transformer-based object detector. Its efficiency comes from the framework design and the Hungarian matching. However, compared to dense supervision detectors like the YOLO series, the Hungarian matching provides much sparser supervision, leading to insufficient model training and difficult to achieve optimal results. To address these issues, we proposed a hierarchical dense positive supervision method based on RT-DETR, named RT-DETRv3. Firstly, we introduce a CNN-based auxiliary branch that provides dense supervision that collaborates with the original decoder to enhance the encoder feature representation. Secondly, to address insufficient decoder training, we propose a novel learning strategy involving self-attention perturbation. This strategy diversifies label assignment for positive samples across multiple query groups, thereby enriching positive supervisions. Additionally, we introduce a shared-weight decoder branch for dense positive supervision to ensure more high-quality queries matching each ground truth. Notably, all aforementioned modules are training-only. We conduct extensive experiments to demonstrate the effectiveness of our approach on COCO val2017. RT-DETRv3 significantly outperforms existing real-time detectors, including the RT-DETR series and the YOLO series. For example, RT-DETRv3-R18 achieves 48.1% AP (+1.6%/+1.4%) compared to RT-DETR-R18/RT-DETRv2-R18, while maintaining the same latency. Furthermore, RT-DETRv3-R101 can attain an impressive 54.6% AP outperforming YOLOv10-X. The code will be released at https://github.com/clxia12/RT-DETRv3.