# Object Detection with Transformers: A Review

arXiv: https://arxiv.org/abs/2306.04670 · HTML: https://mlsanity.com/paper/2306.04670/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 174 of its 239 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 30 of 174)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)

## Built on by (most cited first, 22 of 22)

- [Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Survey](https://mlsanity.com/paper/2310.12904/) (2023)
- [Bridging the Performance Gap between DETR and R-CNN for Graphical Object Detection in Document Images](https://mlsanity.com/paper/2306.13526/) (2023)
- [Superpixel Tokenization for Vision Transformers: Preserving Semantic Integrity in Visual Tokens](https://mlsanity.com/paper/2412.04680/) (2024)
- [A Hybrid Approach for Document Layout Analysis in Document images](https://mlsanity.com/paper/2404.17888/) (2024)
- [Towards End-to-End Semi-Supervised Table Detection with Semantic Aligned Matching Transformer](https://mlsanity.com/paper/2405.00187/) (2024)
- [End-to-End Semi-Supervised approach with Modulated Object Queries for Table Detection in Documents](https://mlsanity.com/paper/2405.04971/) (2024)
- [Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformer](https://mlsanity.com/paper/2407.08460/) (2024)
- [Unsupervised Object Discovery: A Comprehensive Survey and Unified Taxonomy](https://mlsanity.com/paper/2411.00868/) (2024)
- [A multi-modal dataset for insect biodiversity with imagery and DNA at the trap and individual level](https://mlsanity.com/paper/2507.06972/) (2025)
- [UnSupDLA: Towards Unsupervised Document Layout Analysis](https://mlsanity.com/paper/2406.06236/) (2024)
- [MorphXAI: An Explainable Framework for Morphological Analysis of Parasites in Blood Smear Images](https://mlsanity.com/paper/2601.18001/) (2026)
- [Agricultural Object Detection with You Look Only Once (YOLO) Algorithm: A Bibliometric and Systematic Literature Review](https://mlsanity.com/paper/2401.10379/) (2024)
- [Enhanced Bank Check Security: Introducing a Novel Dataset and Transformer-Based Approach for Detection and Verification](https://mlsanity.com/paper/2406.14370/) (2024)
- [Using Interleaved Ensemble Unlearning to Keep Backdoors at Bay for Finetuning Vision Transformers](https://mlsanity.com/paper/2410.01128/) (2024)
- [Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization](https://mlsanity.com/paper/2509.17049/) (2025)
- [VaLID: Verification as Late Integration of Detections for LiDAR-Camera Fusion](https://mlsanity.com/paper/2409.15529/) (2024)
- [Exploring Visual Embedding Spaces Induced by Vision Transformers for Online Auto Parts Marketplaces](https://mlsanity.com/paper/2502.05756/) (2025)
- [Holi-DETR: Holistic Fashion Item Detection Leveraging Contextual Information](https://mlsanity.com/paper/2512.23221/) (2025)
- [MANAR: Memory-augmented Attention with Navigational Abstract Conceptual Representation](https://mlsanity.com/paper/2603.18676/) (2026)
- [Cost-Efficient Multi-Scale Fovea for Semantic-Based Visual Search Attention](https://mlsanity.com/paper/2604.03836/) (2026)
- [ChronoEarth-492K: A Large Scale and Long Horizon Spatiotemporal Hyperspectral Earth Observation Dataset and Benchmark](https://mlsanity.com/paper/2605.15666/) (2026)
- [RailGen: Improving Railway Intrusion Detection via Agent-Guided Small-Scale Foreign Object Generation](https://mlsanity.com/paper/2608.30727/) (2026)

## Abstract (arXiv)

The astounding performance of transformers in natural language processing (NLP) has motivated researchers to explore their applications in computer vision tasks. DEtection TRansformer (DETR) introduces transformers to object detection tasks by reframing detection as a set prediction problem. Consequently, eliminating the need for proposal generation and post-processing steps. Initially, despite competitive performance, DETR suffered from slow training convergence and ineffective detection of smaller objects. However, numerous improvements are proposed to address these issues, leading to substantial improvements in DETR and enabling it to exhibit state-of-the-art performance. To our knowledge, this is the first paper to provide a comprehensive review of 21 recently proposed advancements in the original DETR model. We dive into both the foundational modules of DETR and its recent enhancements, such as modifications to the backbone structure, query design strategies, and refinements to attention mechanisms. Moreover, we conduct a comparative analysis across various detection transformers, evaluating their performance and network architectures. We hope that this study will ignite further interest among researchers in addressing the existing challenges and exploring the application of transformers in the object detection domain. Readers interested in the ongoing developments in detection transformers can refer to our website at: https://github.com/mindgarage-shan/trans_object_detection_survey