# Zero-Shot Aerial Object Detection with Visual Description Regularization

arXiv: https://arxiv.org/abs/2402.18233 · HTML: https://mlsanity.com/paper/2402.18233/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 28 Feb 2024.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark](https://mlsanity.com/paper/1909.00133/) (2019)
- [Few-shot Object Detection via Feature Reweighting](https://mlsanity.com/paper/1812.01866/) (2018)
- [Detection and Tracking Meet Drones Challenge](https://mlsanity.com/paper/2001.06303/) (2020)
- [Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges](https://mlsanity.com/paper/2102.12219/) (2021)
- [Zero-Shot Object Detection](https://mlsanity.com/paper/1804.04340/) (2018)
- [Align Deep Features for Oriented Object Detection](https://mlsanity.com/paper/2008.09397/) (2020)
- [xView: Objects in Context in Overhead Imagery](https://mlsanity.com/paper/1802.07856/) (2018)
- [Oriented RepPoints for Aerial Object Detection](https://mlsanity.com/paper/2105.11111/) (2021)
- [Clustered Object Detection in Aerial Images](https://mlsanity.com/paper/1904.08008/) (2019)
- [Dont Even Look Once: Synthesizing Features for Zero-Shot Detection](https://mlsanity.com/paper/1911.07933/) (2019)
- [Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networks](https://mlsanity.com/paper/1809.03355/) (2018)
- [Density Map Guided Object Detection in Aerial Images](https://mlsanity.com/paper/2004.05520/) (2020)
- [Zero-Shot Object Detection: Learning to Simultaneously Recognize and Localize Novel Concepts](https://mlsanity.com/paper/1803.06049/) (2018)
- [A Generative Approach to Zero-Shot and Few-Shot Action Recognition](https://mlsanity.com/paper/1801.09086/) (2018)
- [Zero-Shot Object Detection by Hybrid Region Embedding](https://mlsanity.com/paper/1805.06157/) (2018)
- [Synthesizing the Unseen for Zero-shot Object Detection](https://mlsanity.com/paper/2010.09425/) (2020)
- [Zero-Shot Visual Recognition via Bidirectional Latent Embedding](https://mlsanity.com/paper/1607.02104/) (2016)
- [GTNet: Generative Transfer Network for Zero-Shot Object Detection](https://mlsanity.com/paper/2001.06812/) (2020)
- [ZEST: Zero-shot Learning from Text Descriptions using Textual Similarity and Visual Summarization](https://mlsanity.com/paper/2010.03276/) (2020)
- [Learning to Detect and Segment for Open Vocabulary Object Detection](https://mlsanity.com/paper/2212.12130/) (2022)
- [Skeleton based Zero Shot Action Recognition in Joint Pose-Language Semantic Space](https://mlsanity.com/paper/1911.11344/) (2019)

## Built on by (most cited first, 10 of 10)

- [OpenRSD: Towards Open-prompts for Object Detection in Remote Sensing Images](https://mlsanity.com/paper/2503.06146/) (2025)
- [InstructSAM: A Training-Free Framework for Instruction-Oriented Remote Sensing Object Recognition](https://mlsanity.com/paper/2505.15818/) (2025)
- [ZoRI: Towards Discriminative Zero-Shot Remote Sensing Instance Segmentation](https://mlsanity.com/paper/2412.12798/) (2024)
- [Open-Vocabulary Object Detection in UAV Imagery: A Review and Future Perspectives](https://mlsanity.com/paper/2507.13359/) (2025)
- [OS-W2S: An Automatic Labeling Engine for Language-Guided Open-Set Aerial Object Detection](https://mlsanity.com/paper/2505.03334/) (2025)
- [Open-Text Aerial Detection: A Unified Framework For Aerial Visual Grounding And Detection](https://mlsanity.com/paper/2602.07827/) (2026)
- [Precision Neural Network Quantization via Learnable Adaptive Modules](https://mlsanity.com/paper/2504.17263/) (2025)
- [MB-ORES: A Multi-Branch Object Reasoner for Visual Grounding in Remote Sensing](https://mlsanity.com/paper/2503.24219/) (2025)
- [Dual Prompt Learning for Adapting Vision-Language Models to Downstream Image-Text Retrieval](https://mlsanity.com/paper/2508.04028/) (2025)
- [Do Open-Vocabulary Detectors Transfer to Aerial Imagery? A Comparative Evaluation](https://mlsanity.com/paper/2601.22164/) (2026)

## Abstract (arXiv)

Existing object detection models are mainly trained on large-scale labeled datasets. However, annotating data for novel aerial object classes is expensive since it is time-consuming and may require expert knowledge. Thus, it is desirable to study label-efficient object detection methods on aerial images. In this work, we propose a zero-shot method for aerial object detection named visual Description Regularization, or DescReg. Concretely, we identify the weak semantic-visual correlation of the aerial objects and aim to address the challenge with prior descriptions of their visual appearance. Instead of directly encoding the descriptions into class embedding space which suffers from the representation gap problem, we propose to infuse the prior inter-class visual similarity conveyed in the descriptions into the embedding learning. The infusion process is accomplished with a newly designed similarity-aware triplet loss which incorporates structured regularization on the representation space. We conduct extensive experiments with three challenging aerial object detection datasets, including DIOR, xView, and DOTA. The results demonstrate that DescReg significantly outperforms the state-of-the-art ZSD methods with complex projection designs and generative frameworks, e.g., DescReg outperforms best reported ZSD method on DIOR by 4.5 mAP on unseen classes and 8.1 in HM. We further show the generalizability of DescReg by integrating it into generative ZSD methods as well as varying the detection architecture.