# Decoupled DETR: Spatially Disentangling Localization and Classification for Improved End-to-End Object Detection

arXiv: https://arxiv.org/abs/2310.15955 · HTML: https://mlsanity.com/paper/2310.15955/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 43 references that are in the map.
- First posted on arXiv 24 Oct 2023.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection](https://mlsanity.com/paper/2006.04388/) (2020)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [TOOD: Task-aligned One-stage Object Detection](https://mlsanity.com/paper/2108.07755/) (2021)
- [Acquisition of Localization Confidence for Accurate Object Detection](https://mlsanity.com/paper/1807.11590/) (2018)
- [Fast Convergence of DETR with Spatially Modulated Co-Attention](https://mlsanity.com/paper/2101.07448/) (2021)
- [Scale-Aware Trident Networks for Object Detection](https://mlsanity.com/paper/1901.01892/) (2019)
- [DETRs with Hybrid Matching](https://mlsanity.com/paper/2207.13080/) (2022)
- [Feature Selective Anchor-Free Module for Single-Shot Object Detection](https://mlsanity.com/paper/1903.00621/) (2019)
- [FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation](https://mlsanity.com/paper/2303.01237/) (2023)
- [Rethinking Classification and Localization for Object Detection](https://mlsanity.com/paper/1904.06493/) (2019)
- [FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting](https://mlsanity.com/paper/2109.02974/) (2021)
- [Revisiting the Sibling Head in Object Detector](https://mlsanity.com/paper/2003.07540/) (2020)
- [VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation](https://mlsanity.com/paper/2303.08340/) (2023)
- [Accelerating DETR Convergence via Semantic-Aligned Matching](https://mlsanity.com/paper/2203.06883/) (2022)

## Built on by (most cited first, 8 of 8)

- [DAMap: Distance-aware MapNet for High Quality HD Map Construction](https://mlsanity.com/paper/2510.22675/) (2025)
- [DSD-DA: Distillation-based Source Debiasing for Domain Adaptive Object Detection](https://mlsanity.com/paper/2311.10437/) (2023)
- [DPDETR: Decoupled Position Detection Transformer for Infrared-Visible Object Detection](https://mlsanity.com/paper/2408.06123/) (2024)
- [CoreNet: Conflict Resolution Network for Point-Pixel Misalignment and Sub-Task Suppression of 3D LiDAR-Camera Object Detection](https://mlsanity.com/paper/2501.06550/) (2025)
- [Towards Large-scale Masked Face Recognition](https://mlsanity.com/paper/2310.16364/) (2023)
- [Mr. DETR++: Instructive Multi-Route Training for Detection Transformers with Mixture-of-Experts](https://mlsanity.com/paper/2412.10028/) (2024)
- [CrowdQuery: Density-Guided Query Module for Enhanced 2D and 3D Detection in Crowded Scenes](https://mlsanity.com/paper/2509.08738/) (2025)
- [CellDETR: A Detection-Guided Framework for Scalable Cell Representation Learning from Histopathology Images](https://mlsanity.com/paper/2606.29463/) (2026)

## Abstract (arXiv)

The introduction of DETR represents a new paradigm for object detection. However, its decoder conducts classification and box localization using shared queries and cross-attention layers, leading to suboptimal results. We observe that different regions of interest in the visual feature map are suitable for performing query classification and box localization tasks, even for the same object. Salient regions provide vital information for classification, while the boundaries around them are more favorable for box regression. Unfortunately, such spatial misalignment between these two tasks greatly hinders DETR's training. Therefore, in this work, we focus on decoupling localization and classification tasks in DETR. To achieve this, we introduce a new design scheme called spatially decoupled DETR (SD-DETR), which includes a task-aware query generation module and a disentangled feature learning process. We elaborately design the task-aware query initialization process and divide the cross-attention block in the decoder to allow the task-aware queries to match different visual regions. Meanwhile, we also observe that the prediction misalignment problem for high classification confidence and precise localization exists, so we propose an alignment loss to further guide the spatially decoupled DETR training. Through extensive experiments, we demonstrate that our approach achieves a significant improvement in MSCOCO datasets compared to previous work. For instance, we improve the performance of Conditional DETR by 4.5 AP. By spatially disentangling the two tasks, our method overcomes the misalignment problem and greatly improves the performance of DETR for object detection.