# Anchor DETR: Query Design for Transformer-Based Object Detection

arXiv: https://arxiv.org/abs/2109.07107 · HTML: https://mlsanity.com/paper/2109.07107/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 28 references that are in the map.
- First posted on arXiv 15 Sep 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [Efficient Attention: Attention with Linear Complexities](https://mlsanity.com/paper/1812.01243/) (2018)
- [DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution](https://mlsanity.com/paper/2006.02334/) (2020)
- [Fast Convergence of DETR with Spatially Modulated Co-Attention](https://mlsanity.com/paper/2101.07448/) (2021)
- [You Only Look One-level Feature](https://mlsanity.com/paper/2103.09460/) (2021)
- [Luna: Linear Unified Nested Attention](https://mlsanity.com/paper/2106.01540/) (2021)
- [End-to-End Object Detection with Fully Convolutional Network](https://mlsanity.com/paper/2012.03544/) (2020)

## Built on by (most cited first, 30 of 30)

- [AO2-DETR: Arbitrary-Oriented Object Detection Transformer](https://mlsanity.com/paper/2205.12785/) (2022)
- [TAPTR: Tracking Any Point with Transformers as Detection](https://mlsanity.com/paper/2403.13042/) (2024)
- [LATR: 3D Lane Detection from Monocular Images with Transformer](https://mlsanity.com/paper/2308.04583/) (2023)
- [Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2209.13306/) (2022)
- [Understanding 3D Object Interaction from a Single Image](https://mlsanity.com/paper/2305.09664/) (2023)
- [Dual DETRs for Multi-Label Temporal Action Detection](https://mlsanity.com/paper/2404.00653/) (2024)
- [Focal-PETR: Embracing Foreground for Efficient Multi-Camera 3D Object Detection](https://mlsanity.com/paper/2212.05505/) (2022)
- [DiffTAD: Temporal Action Detection with Proposal Denoising Diffusion](https://mlsanity.com/paper/2303.14863/) (2023)
- [3DPPE: 3D Point Positional Encoding for Multi-Camera 3D Object Detection Transformers](https://mlsanity.com/paper/2211.14710/) (2022)
- [Transformers in Small Object Detection: A Benchmark and Survey of State-of-the-Art](https://mlsanity.com/paper/2309.04902/) (2023)
- [Learning Dynamic Style Kernels for Artistic Style Transfer](https://mlsanity.com/paper/2304.00414/) (2023)
- [VGSG: Vision-Guided Semantic-Group Network for Text-based Person Search](https://mlsanity.com/paper/2311.07514/) (2023)
- [MoGDE: Boosting Mobile Monocular 3D Object Detection with Ground Depth Estimation](https://mlsanity.com/paper/2303.13561/) (2023)
- [An Efficient Transformer for Simultaneous Learning of BEV and Lane Representations in 3D Lane Detection](https://mlsanity.com/paper/2306.04927/) (2023)
- [Rethinking Remote Sensing Change Detection With A Mask View](https://mlsanity.com/paper/2406.15320/) (2024)
- [Representation Shift: Unifying Token Compression with FlashAttention](https://mlsanity.com/paper/2508.00367/) (2025)
- [EAN-MapNet: Efficient Vectorized HD Map Construction with Anchor Neighborhoods](https://mlsanity.com/paper/2402.18278/) (2024)
- [Proposal-Contrastive Pretraining for Object Detection from Fewer Data](https://mlsanity.com/paper/2310.16835/) (2023)
- [Learning Saliency From Fixations](https://mlsanity.com/paper/2311.14073/) (2023)
- [Historical Astronomical Diagrams Decomposition in Geometric Primitives](https://mlsanity.com/paper/2403.08721/) (2024)
- [SEFormer: Structure Embedding Transformer for 3D Object Detection](https://mlsanity.com/paper/2209.01745/) (2022)
- [KS-DETR: Knowledge Sharing in Attention Learning for Detection Transformer](https://mlsanity.com/paper/2302.11208/) (2023)
- [MomentMix Augmentation with Length-Aware DETR for Temporally Robust Moment Retrieval](https://mlsanity.com/paper/2412.20816/) (2024)
- [Comprehension of Multilingual Expressions Referring to Target Objects in Visual Inputs](https://mlsanity.com/paper/2511.11427/) (2025)
- [Towards Few-Annotation Learning in Computer Vision: Application to Image Classification and Object Detection tasks](https://mlsanity.com/paper/2311.04888/) (2023)
- [Better Sampling, towards Better End-to-end Small Object Detection](https://mlsanity.com/paper/2407.06127/) (2024)
- [Cross Resolution Encoding-Decoding For Detection Transformers](https://mlsanity.com/paper/2410.04088/) (2024)
- [A lightweight model FDM-YOLO for small target improvement based on YOLOv8](https://mlsanity.com/paper/2503.04452/) (2025)
- [Beyond Caption-Based Queries for Video Moment Retrieval](https://mlsanity.com/paper/2603.02363/) (2026)
- [CAM3DNet: Comprehensively mining the multi-scale features for 3D Object Detection with Multi-View Cameras](https://mlsanity.com/paper/2604.17024/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel query design for the transformer-based object detection. In previous transformer-based detectors, the object queries are a set of learned embeddings. However, each learned embedding does not have an explicit physical meaning and we cannot explain where it will focus on. It is difficult to optimize as the prediction slot of each object query does not have a specific mode. In other words, each object query will not focus on a specific region. To solved these problems, in our query design, object queries are based on anchor points, which are widely used in CNN-based detectors. So each object query focuses on the objects near the anchor point. Moreover, our query design can predict multiple objects at one position to solve the difficulty: "one region, multiple objects". In addition, we design an attention variant, which can reduce the memory cost while achieving similar or better performance than the standard attention in DETR. Thanks to the query design and the attention variant, the proposed detector that we called Anchor DETR, can achieve better performance and run faster than the DETR with 10× fewer training epochs. For example, it achieves 44.2 AP with 19 FPS on the MSCOCO dataset when using the ResNet50-DC5 feature for training 50 epochs. Extensive experiments on the MSCOCO benchmark prove the effectiveness of the proposed methods. Code is available at urlhttps://github.com/megvii-research/AnchorDETR.