# Efficient Object Detection in Optical Remote Sensing Imagery via Attention-based Feature Distillation arXiv: https://arxiv.org/abs/2310.18676 ยท HTML: https://mlsanity.com/paper/2310.18676/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. - Builds on 42 of its 59 references that are in the map. - First posted on arXiv 28 Oct 2023. ## Placed in - Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/) ## Builds on (most cited first, 30 of 42) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015) - [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018) - [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017) - [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015) - [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015) - [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017) - [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019) - [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014) - [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017) - [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019) - [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019) - [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019) - [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014) - [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017) - [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019) - [Similarity-Preserving Knowledge Distillation](https://mlsanity.com/paper/1907.09682/) (2019) - [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019) - [GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond](https://mlsanity.com/paper/1904.11492/) (2019) - [Dynamic Network Surgery for Efficient DNNs](https://mlsanity.com/paper/1608.04493/) (2016) - [R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object](https://mlsanity.com/paper/1908.05612/) (2019) - [Distilling Object Detectors with Fine-grained Feature Imitation](https://mlsanity.com/paper/1906.03609/) (2019) - [Focal and Global Knowledge Distillation for Detectors](https://mlsanity.com/paper/2111.11837/) (2021) - [UIU-Net: U-Net in U-Net for Infrared Small Object Detection](https://mlsanity.com/paper/2212.00968/) (2022) - [General Instance Distillation for Object Detection](https://mlsanity.com/paper/2103.02340/) (2021) - [Distilling Object Detectors via Decoupled Features](https://mlsanity.com/paper/2103.14475/) (2021) - [Dynamic Refinement Network for Oriented and Densely Packed Object Detection](https://mlsanity.com/paper/2005.09973/) (2020) - [Wasserstein Contrastive Representation Distillation](https://mlsanity.com/paper/2012.08674/) (2020) ## Built on by (most cited first, 6 of 6) - [Diff-Mosaic: Augmenting Realistic Representations in Infrared Small Target Detection via Diffusion Prior](https://mlsanity.com/paper/2406.00632/) (2024) - [Applications of Knowledge Distillation in Remote Sensing: A Survey](https://mlsanity.com/paper/2409.12111/) (2024) - [HyperKD: Distilling Cross-Spectral Knowledge in Masked Autoencoders via Inverse Domain Shift with Spatial-Aware Masking and Specialized Loss](https://mlsanity.com/paper/2508.09453/) (2025) - [Dual-Stream Spectral Decoupling Distillation for Remote Sensing Object Detection](https://mlsanity.com/paper/2512.04413/) (2025) - [HMR-Net: Hierarchical Modular Routing for Cross-Domain Object Detection in Aerial Images](https://mlsanity.com/paper/2604.18866/) (2026) - [Machine Learning under Imperfect Data: Challenges and Methods](https://mlsanity.com/paper/2609.13914/) (2026) ## Abstract (arXiv) Efficient object detection methods have recently received great attention in remote sensing. Although deep convolutional networks often have excellent detection accuracy, their deployment on resource-limited edge devices is difficult. Knowledge distillation (KD) is a strategy for addressing this issue since it makes models lightweight while maintaining accuracy. However, existing KD methods for object detection have encountered two constraints. First, they discard potentially important background information and only distill nearby foreground regions. Second, they only rely on the global context, which limits the student detector's ability to acquire local information from the teacher detector. To address the aforementioned challenges, we propose Attention-based Feature Distillation (AFD), a new KD approach that distills both local and global information from the teacher detector. To enhance local distillation, we introduce a multi-instance attention mechanism that effectively distinguishes between background and foreground elements. This approach prompts the student detector to focus on the pertinent channels and pixels, as identified by the teacher detector. Local distillation lacks global information, thus attention global distillation is proposed to reconstruct the relationship between various pixels and pass it from teacher to student detector. The performance of AFD is evaluated on two public aerial image benchmarks, and the evaluation results demonstrate that AFD in object detection can attain the performance of other state-of-the-art models while being efficient.