# AO2-DETR: Arbitrary-Oriented Object Detection Transformer

arXiv: https://arxiv.org/abs/2205.12785 · HTML: https://mlsanity.com/paper/2205.12785/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 60 references that are in the map.
- First posted on arXiv 25 May 2022.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [RepPoints: Point Set Representation for Object Detection](https://mlsanity.com/paper/1904.11490/) (2019)
- [Oriented R-CNN for Object Detection](https://mlsanity.com/paper/2108.05699/) (2021)
- [ReDet: A Rotation-equivariant Detector for Aerial Object Detection](https://mlsanity.com/paper/2103.07733/) (2021)
- [R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object](https://mlsanity.com/paper/1908.05612/) (2019)
- [Align Deep Features for Oriented Object Detection](https://mlsanity.com/paper/2008.09397/) (2020)
- [Arbitrary-Oriented Scene Text Detection via Rotation Proposals](https://mlsanity.com/paper/1703.01086/) (2017)
- [Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss](https://mlsanity.com/paper/2101.11952/) (2021)
- [Gliding vertex on the horizontal bounding box for multi-oriented object detection](https://mlsanity.com/paper/1911.09358/) (2019)
- [Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergence](https://mlsanity.com/paper/2106.01883/) (2021)
- [Single-Shot Refinement Neural Network for Object Detection](https://mlsanity.com/paper/1711.06897/) (2017)
- [IoU Loss for 2D/3D Object Detection](https://mlsanity.com/paper/1908.03851/) (2019)
- [MMRotate: A Rotated Object Detection Benchmark using PyTorch](https://mlsanity.com/paper/2204.13317/) (2022)
- [Oriented RepPoints for Aerial Object Detection](https://mlsanity.com/paper/2105.11111/) (2021)
- [Rotation-Sensitive Regression for Oriented Scene Text Detection](https://mlsanity.com/paper/1803.05265/) (2018)
- [Dynamic Refinement Network for Oriented and Densely Packed Object Detection](https://mlsanity.com/paper/2005.09973/) (2020)
- [Dynamic Anchor Learning for Arbitrary-Oriented Object Detection](https://mlsanity.com/paper/2012.04150/) (2020)
- [Precise Detection in Densely Packed Scenes](https://mlsanity.com/paper/1904.00853/) (2019)
- [What Makes for End-to-End Object Detection?](https://mlsanity.com/paper/2012.05780/) (2020)

## Built on by (most cited first, 30 of 55)

- [Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model](https://mlsanity.com/paper/2208.03987/) (2022)
- [Large Selective Kernel Network for Remote Sensing Object Detection](https://mlsanity.com/paper/2303.09030/) (2023)
- [MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining](https://mlsanity.com/paper/2403.13430/) (2024)
- [Transformers in Remote Sensing: A Survey](https://mlsanity.com/paper/2209.01206/) (2022)
- [Poly Kernel Inception Network for Remote Sensing Detection](https://mlsanity.com/paper/2403.06258/) (2024)
- [LSKNet: A Foundation Lightweight Backbone for Remote Sensing](https://mlsanity.com/paper/2403.11735/) (2024)
- [ARS-DETR: Aspect Ratio-Sensitive Detection Transformer for Aerial Oriented Object Detection](https://mlsanity.com/paper/2303.04989/) (2023)
- [Graph Transformer GANs for Graph-Constrained House Generation](https://mlsanity.com/paper/2303.08225/) (2023)
- [Transformers in Small Object Detection: A Benchmark and Survey of State-of-the-Art](https://mlsanity.com/paper/2309.04902/) (2023)
- [Popeye: A Unified Visual-Language Model for Multi-Source Ship Detection from Remote Sensing Imagery](https://mlsanity.com/paper/2403.03790/) (2024)
- [Strip R-CNN: Large Strip Convolution for Remote Sensing Object Detection](https://mlsanity.com/paper/2501.03775/) (2025)
- [SOOD++: Leveraging Unlabeled Data to Boost Oriented Object Detection](https://mlsanity.com/paper/2407.01016/) (2024)
- [A Review of Transformer-Based Models for Computer Vision Tasks: Capturing Global Context and Spatial Relationships](https://mlsanity.com/paper/2408.15178/) (2024)
- [OrientedFormer: An End-to-End Transformer-Based Oriented Object Detector in Remote Sensing Images](https://mlsanity.com/paper/2409.19648/) (2024)
- [Frequency-Dynamic Attention Modulation for Dense Prediction](https://mlsanity.com/paper/2507.12006/) (2025)
- [SSD-MonoDETR: Supervised Scale-aware Deformable Transformer for Monocular 3D Object Detection](https://mlsanity.com/paper/2305.07270/) (2023)
- [Hausdorff Distance Matching with Adaptive Query Denoising for Rotated Detection Transformer](https://mlsanity.com/paper/2305.07598/) (2023)
- [Background Semantics Matter: Cross-Task Feature Exchange Network for Clustered Infrared Small Target Detection](https://mlsanity.com/paper/2407.20078/) (2024)
- [Oriented object detection in optical remote sensing images using deep learning: a survey](https://mlsanity.com/paper/2302.10473/) (2023)
- [Learning Remote Sensing Object Detection with Single Point Supervision](https://mlsanity.com/paper/2305.14141/) (2023)
- [LEGNet: A Lightweight Edge-Gaussian Network for Low-Quality Remote Sensing Image Object Detection](https://mlsanity.com/paper/2503.14012/) (2025)
- [Edge-guided Representation Learning for Underwater Object Detection](https://mlsanity.com/paper/2306.00440/) (2023)
- [A Gated Cross-domain Collaborative Network for Underwater Object Detection](https://mlsanity.com/paper/2306.14141/) (2023)
- [Few-shot Object Detection in Remote Sensing: Lifting the Curse of Incompletely Annotated Novel Objects](https://mlsanity.com/paper/2309.10588/) (2023)
- [RQFormer: Rotated Query Transformer for End-to-End Oriented Object Detection](https://mlsanity.com/paper/2311.17629/) (2023)
- [Graph Transformer GANs with Graph Masked Modeling for Architectural Layout Generation](https://mlsanity.com/paper/2401.07721/) (2024)
- [MutDet: Mutually Optimizing Pre-training for Remote Sensing Object Detection](https://mlsanity.com/paper/2407.09920/) (2024)
- [DPDETR: Decoupled Position Detection Transformer for Infrared-Visible Object Detection](https://mlsanity.com/paper/2408.06123/) (2024)
- [PETDet: Proposal Enhancement for Two-Stage Fine-Grained Object Detection](https://mlsanity.com/paper/2312.10515/) (2023)
- [Cross-modal Offset-guided Dynamic Alignment and Fusion for Weakly Aligned UAV Object Detection](https://mlsanity.com/paper/2506.16737/) (2025)

## Abstract (arXiv)

Arbitrary-oriented object detection (AOOD) is a challenging task to detect objects in the wild with arbitrary orientations and cluttered arrangements. Existing approaches are mainly based on anchor-based boxes or dense points, which rely on complicated hand-designed processing steps and inductive bias, such as anchor generation, transformation, and non-maximum suppression reasoning. Recently, the emerging transformer-based approaches view object detection as a direct set prediction problem that effectively removes the need for hand-designed components and inductive biases. In this paper, we propose an Arbitrary-Oriented Object DEtection TRansformer framework, termed AO2-DETR, which comprises three dedicated components. More precisely, an oriented proposal generation mechanism is proposed to explicitly generate oriented proposals, which provides better positional priors for pooling features to modulate the cross-attention in the transformer decoder. An adaptive oriented proposal refinement module is introduced to extract rotation-invariant region features and eliminate the misalignment between region features and objects. And a rotation-aware set matching loss is used to ensure the one-to-one matching process for direct set prediction without duplicate predictions. Our method considerably simplifies the overall pipeline and presents a new AOOD paradigm. Comprehensive experiments on several challenging datasets show that our method achieves superior performance on the AOOD task.