# SO-DETR: Leveraging Dual-Domain Features and Knowledge Distillation for Small Object Detection

arXiv: https://arxiv.org/abs/2504.11470 · HTML: https://mlsanity.com/paper/2504.11470/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 11 Apr 2025.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 24 of 24)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [DETRs Beat YOLOs on Real-time Object Detection](https://mlsanity.com/paper/2304.08069/) (2023)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [YOLOv10: Real-Time End-to-End Object Detection](https://mlsanity.com/paper/2405.14458/) (2024)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [YOLOv11: An Overview of the Key Architectural Enhancements](https://mlsanity.com/paper/2410.17725/) (2024)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [Detection and Tracking Meet Drones Challenge](https://mlsanity.com/paper/2001.06303/) (2020)
- [Rethinking Vision Transformers for MobileNet Size and Speed](https://mlsanity.com/paper/2212.08059/) (2022)
- [Clustered Object Detection in Aerial Images](https://mlsanity.com/paper/1904.08008/) (2019)
- [QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection](https://mlsanity.com/paper/2103.09136/) (2021)
- [RFLA: Gaussian Receptive Field based Label Assignment for Tiny Object Detection](https://mlsanity.com/paper/2208.08738/) (2022)
- [SIoU Loss: More Powerful Learning for Bounding Box Regression](https://mlsanity.com/paper/2205.12740/) (2022)
- [Dynamic Coarse-to-Fine Learning for Oriented Tiny Object Detection](https://mlsanity.com/paper/2304.08876/) (2023)
- [DETRDistill: A Universal Knowledge Distillation Framework for DETR-families](https://mlsanity.com/paper/2211.10156/) (2022)
- [UAV-DETR: Efficient End-to-End Object Detection for Unmanned Aerial Vehicle Imagery](https://mlsanity.com/paper/2501.01855/) (2025)
- [KD-DETR: Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling](https://mlsanity.com/paper/2211.08071/) (2022)
- [HIC-YOLOv5: Improved YOLOv5 For Small Object Detection](https://mlsanity.com/paper/2309.16393/) (2023)
- [Inner-IoU: More Effective Intersection over Union Loss with Auxiliary Bounding Box](https://mlsanity.com/paper/2311.02877/) (2023)

## Built on by (most cited first, 3 of 3)

- [Architectural Insights into Knowledge Distillation for Object Detection: A Comprehensive Review](https://mlsanity.com/paper/2508.03317/) (2025)
- [RareSpot+: A Benchmark, Model, and Active Learning Framework for Small and Rare Wildlife in Aerial Imagery](https://mlsanity.com/paper/2604.20000/) (2026)
- [TinyFormer: Preserving Tiny Objects in YOLO-DETR Hybrid Real-time Detectors](https://mlsanity.com/paper/2605.25046/) (2026)

## Abstract (arXiv)

Detection Transformer-based methods have achieved significant advancements in general object detection. However, challenges remain in effectively detecting small objects. One key difficulty is that existing encoders struggle to efficiently fuse low-level features. Additionally, the query selection strategies are not effectively tailored for small objects. To address these challenges, this paper proposes an efficient model, Small Object Detection Transformer (SO-DETR). The model comprises three key components: a dual-domain hybrid encoder, an enhanced query selection mechanism, and a knowledge distillation strategy. The dual-domain hybrid encoder integrates spatial and frequency domains to fuse multi-scale features effectively. This approach enhances the representation of high-resolution features while maintaining relatively low computational overhead. The enhanced query selection mechanism optimizes query initialization by dynamically selecting high-scoring anchor boxes using expanded IoU, thereby improving the allocation of query resources. Furthermore, by incorporating a lightweight backbone network and implementing a knowledge distillation strategy, we develop an efficient detector for small objects. Experimental results on the VisDrone-2019-DET and UAVVaste datasets demonstrate that SO-DETR outperforms existing methods with similar computational demands. The project page is available at https://github.com/ValiantDiligent/SO_DETR.