# MutDet: Mutually Optimizing Pre-training for Remote Sensing Object Detection

arXiv: https://arxiv.org/abs/2407.09920 · HTML: https://mlsanity.com/paper/2407.09920/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 47 references that are in the map.
- First posted on arXiv 13 Jul 2024.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark](https://mlsanity.com/paper/1909.00133/) (2019)
- [Oriented R-CNN for Object Detection](https://mlsanity.com/paper/2108.05699/) (2021)
- [ReDet: A Rotation-equivariant Detector for Aerial Object Detection](https://mlsanity.com/paper/2103.07733/) (2021)
- [FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagery](https://mlsanity.com/paper/2103.05569/) (2021)
- [Knowledge Distillation with the Reused Teacher Classifier](https://mlsanity.com/paper/2203.14001/) (2022)
- [MMRotate: A Rotated Object Detection Benchmark using PyTorch](https://mlsanity.com/paper/2204.13317/) (2022)
- [Aligning Pretraining for Detection via Object-Level Contrastive Learning](https://mlsanity.com/paper/2106.02637/) (2021)
- [Anchor-free Oriented Proposal Generator for Object Detection](https://mlsanity.com/paper/2110.01931/) (2021)
- [Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignment](https://mlsanity.com/paper/2207.13085/) (2022)
- [DETReg: Unsupervised Pretraining with Region Priors for Object Detection](https://mlsanity.com/paper/2106.04550/) (2021)
- [The KFIoU Loss for Rotated Object Detection](https://mlsanity.com/paper/2201.12558/) (2022)
- [Localization Distillation for Dense Object Detection](https://mlsanity.com/paper/2102.12252/) (2021)
- [AO2-DETR: Arbitrary-Oriented Object Detection Transformer](https://mlsanity.com/paper/2205.12785/) (2022)
- [CrossKD: Cross-Head Knowledge Distillation for Object Detection](https://mlsanity.com/paper/2306.11369/) (2023)
- [Phase-Shifting Coder: Predicting Accurate Orientation in Oriented Object Detection](https://mlsanity.com/paper/2211.06368/) (2022)
- [Mutual Contrastive Learning for Visual Representation Learning](https://mlsanity.com/paper/2104.12565/) (2021)
- [Revisiting the Transferability of Supervised Pretraining: an MLP Perspective](https://mlsanity.com/paper/2112.00496/) (2021)
- [Detecting Rotated Objects as Gaussian Distributions and Its 3-D Generalization](https://mlsanity.com/paper/2209.10839/) (2022)
- [DETRDistill: A Universal Knowledge Distillation Framework for DETR-families](https://mlsanity.com/paper/2211.10156/) (2022)

## Built on by (most cited first, 5 of 5)

- [OpenRSD: Towards Open-prompts for Object Detection in Remote Sensing Images](https://mlsanity.com/paper/2503.06146/) (2025)
- [Fourier Angle Alignment for Oriented Object Detection in Remote Sensing](https://mlsanity.com/paper/2602.23790/) (2026)
- [Digital-to-Physical Transfer of Adversarial Patches for Aerial Vehicle Detection](https://mlsanity.com/paper/2606.00159/) (2026)
- [Advancing Metallic Surface Defect Detection via Anomaly-Guided Pretraining on a Large Industrial Dataset](https://mlsanity.com/paper/2509.18919/) (2025)
- [FGAA-FPN: Foreground-Guided Angle-Aware Feature Pyramid Network for Oriented Object Detection](https://mlsanity.com/paper/2602.10710/) (2026)

## Abstract (arXiv)

Detection pre-training methods for the DETR series detector have been extensively studied in natural scenes, e.g., DETReg. However, the detection pre-training remains unexplored in remote sensing scenes. In existing pre-training methods, alignment between object embeddings extracted from a pre-trained backbone and detector features is significant. However, due to differences in feature extraction methods, a pronounced feature discrepancy still exists and hinders the pre-training performance. The remote sensing images with complex environments and more densely distributed objects exacerbate the discrepancy. In this work, we propose a novel Mutually optimizing pre-training framework for remote sensing object Detection, dubbed as MutDet. In MutDet, we propose a systemic solution against this challenge. Firstly, we propose a mutual enhancement module, which fuses the object embeddings and detector features bidirectionally in the last encoder layer, enhancing their information interaction.Secondly, contrastive alignment loss is employed to guide this alignment process softly and simultaneously enhances detector features' discriminativity. Finally, we design an auxiliary siamese head to mitigate the task gap arising from the introduction of enhancement module. Comprehensive experiments on various settings show new state-of-the-art transfer performance. The improvement is particularly pronounced when data quantity is limited. When using 10% of the DIOR-R data, MutDet improves DetReg by 6.1% in AP50. Codes and models are available at: https://github.com/floatingstarZ/MutDet.