# OrientedFormer: An End-to-End Transformer-Based Oriented Object Detector in Remote Sensing Images

arXiv: https://arxiv.org/abs/2409.19648 · HTML: https://mlsanity.com/paper/2409.19648/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 66 references that are in the map.
- First posted on arXiv 29 Sep 2024.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [Oriented R-CNN for Object Detection](https://mlsanity.com/paper/2108.05699/) (2021)
- [Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges](https://mlsanity.com/paper/2102.12219/) (2021)
- [ReDet: A Rotation-equivariant Detector for Aerial Object Detection](https://mlsanity.com/paper/2103.07733/) (2021)
- [R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object](https://mlsanity.com/paper/1908.05612/) (2019)
- [Align Deep Features for Oriented Object Detection](https://mlsanity.com/paper/2008.09397/) (2020)
- [Fast Convergence of DETR with Spatially Modulated Co-Attention](https://mlsanity.com/paper/2101.07448/) (2021)
- [Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss](https://mlsanity.com/paper/2101.11952/) (2021)
- [Gliding vertex on the horizontal bounding box for multi-oriented object detection](https://mlsanity.com/paper/1911.09358/) (2019)
- [IoU Loss for 2D/3D Object Detection](https://mlsanity.com/paper/1908.03851/) (2019)
- [MMRotate: A Rotated Object Detection Benchmark using PyTorch](https://mlsanity.com/paper/2204.13317/) (2022)
- [Anchor-free Oriented Proposal Generator for Object Detection](https://mlsanity.com/paper/2110.01931/) (2021)
- [Oriented RepPoints for Aerial Object Detection](https://mlsanity.com/paper/2105.11111/) (2021)
- [Large Selective Kernel Network for Remote Sensing Object Detection](https://mlsanity.com/paper/2303.09030/) (2023)
- [AdaMixer: A Fast-Converging Query-Based Object Detector](https://mlsanity.com/paper/2203.16507/) (2022)
- [Dense Distinct Query for End-to-End Object Detection](https://mlsanity.com/paper/2303.12776/) (2023)
- [SCRDet++: Detecting Small, Cluttered and Rotated Objects via Instance-Level Feature Denoising and Rotation Loss Smoothing](https://mlsanity.com/paper/2004.13316/) (2020)
- [PIoU Loss: Towards Accurate Oriented Object Detection in Complex Environments](https://mlsanity.com/paper/2007.09584/) (2020)

## Built on by (most cited first, 10 of 10)

- [More Clear, More Flexible, More Precise: A Comprehensive Oriented Object Detection benchmark for UAV](https://mlsanity.com/paper/2504.20032/) (2025)
- [Real-Time Oriented Object Detection Transformer in Remote Sensing Images](https://mlsanity.com/paper/2603.15497/) (2026)
- [RSRWKV: A Linear-Complexity 2D Attention Mechanism for Efficient Remote Sensing Vision Task](https://mlsanity.com/paper/2503.20382/) (2025)
- [Explore Intrinsic Geometry for Query-based Tiny and Oriented Object Detector with Momentum-based Bipartite Matching](https://mlsanity.com/paper/2602.13728/) (2026)
- [Multi-Grained Vision-Language Alignment for Domain Generalized Person Re-Identification](https://mlsanity.com/paper/2603.14012/) (2026)
- [HMR-Net: Hierarchical Modular Routing for Cross-Domain Object Detection in Aerial Images](https://mlsanity.com/paper/2604.18866/) (2026)
- [A novel YOLO26-MoE optimized by an LLM agent for insulator fault detection considering UAV images](https://mlsanity.com/paper/2605.19595/) (2026)
- [BMCR: Adaptive Backbone Module Composition via Reinforcement Learning for Remote Sensing Object Detection](https://mlsanity.com/paper/2606.05586/) (2026)
- [Geometry Meets Semantics: Fractional Gradient Stabilization for Semantic-Driven Bounding Box Optimization in Visual Detection Tasks](https://mlsanity.com/paper/2607.23530/) (2026)
- [A Unified Framework and Dataset for Oriented Object Visual Grounding in Remote Sensing](https://mlsanity.com/paper/2609.28230/) (2026)

## Abstract (arXiv)

Oriented object detection in remote sensing images is a challenging task due to objects being distributed in multi-orientation. Recently, end-to-end transformer-based methods have achieved success by eliminating the need for post-processing operators compared to traditional CNN-based methods. However, directly extending transformers to oriented object detection presents three main issues: 1) objects rotate arbitrarily, necessitating the encoding of angles along with position and size; 2) the geometric relations of oriented objects are lacking in self-attention, due to the absence of interaction between content and positional queries; and 3) oriented objects cause misalignment, mainly between values and positional queries in cross-attention, making accurate classification and localization difficult. In this paper, we propose an end-to-end transformer-based oriented object detector, consisting of three dedicated modules to address these issues. First, Gaussian positional encoding is proposed to encode the angle, position, and size of oriented boxes using Gaussian distributions. Second, Wasserstein self-attention is proposed to introduce geometric relations and facilitate interaction between content and positional queries by utilizing Gaussian Wasserstein distance scores. Third, oriented cross-attention is proposed to align values and positional queries by rotating sampling points around the positional query according to their angles. Experiments on six datasets DIOR-R, a series of DOTA, HRSC2016 and ICDAR2015 show the effectiveness of our approach. Compared with previous end-to-end detectors, the OrientedFormer gains 1.16 and 1.21 AP50 on DIOR-R and DOTA-v1.0 respectively, while reducing training epochs from 3× to 1×. The codes are available at https://github.com/wokaikaixinxin/ai4rs and https://github.com/wokaikaixinxin/OrientedFormer.