# Localization Distillation for Dense Object Detection

arXiv: https://arxiv.org/abs/2102.12252 · HTML: https://mlsanity.com/paper/2102.12252/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 54 of its 73 references that are in the map.
- First posted on arXiv 24 Feb 2021.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 30 of 54)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Improved Knowledge Distillation via Teacher Assistant](https://mlsanity.com/paper/1902.03393/) (2019)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection](https://mlsanity.com/paper/2006.04388/) (2020)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [Libra R-CNN: Towards Balanced Learning for Object Detection](https://mlsanity.com/paper/1904.02701/) (2019)
- [VarifocalNet: An IoU-aware Dense Object Detector](https://mlsanity.com/paper/2008.13367/) (2020)

## Built on by (most cited first, 30 of 55)

- [D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement](https://mlsanity.com/paper/2410.13842/) (2024)
- [BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2211.09386/) (2022)
- [CrossKD: Cross-Head Knowledge Distillation for Object Detection](https://mlsanity.com/paper/2306.11369/) (2023)
- [PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient](https://mlsanity.com/paper/2207.02039/) (2022)
- [DETRDistill: A Universal Knowledge Distillation Framework for DETR-families](https://mlsanity.com/paper/2211.10156/) (2022)
- [ViTKD: Practical Guidelines for ViT feature knowledge distillation](https://mlsanity.com/paper/2209.02432/) (2022)
- [Visual Tuning](https://mlsanity.com/paper/2305.06061/) (2023)
- [Teacher-Student Architecture for Knowledge Distillation: A Survey](https://mlsanity.com/paper/2308.04268/) (2023)
- [Localization Distillation for Object Detection](https://mlsanity.com/paper/2204.05957/) (2022)
- [SemiReward: A General Reward Model for Semi-supervised Learning](https://mlsanity.com/paper/2310.03013/) (2023)
- [Strip R-CNN: Large Strip Convolution for Remote Sensing Object Detection](https://mlsanity.com/paper/2501.03775/) (2025)
- [VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detection](https://mlsanity.com/paper/2506.04623/) (2025)
- [DistilPose: Tokenized Pose Regression with Heatmap Distillation](https://mlsanity.com/paper/2303.02455/) (2023)
- [Decomposed Cross-modal Distillation for RGB-based Temporal Action Detection](https://mlsanity.com/paper/2303.17285/) (2023)
- [On the Road to Portability: Compressing End-to-End Motion Planner for Autonomous Driving](https://mlsanity.com/paper/2403.01238/) (2024)
- [Spatial Self-Distillation for Object Detection with Inaccurate Bounding Boxes](https://mlsanity.com/paper/2307.12101/) (2023)
- [StereoDistill: Pick the Cream from LiDAR for Distilling Stereo-based 3D Object Detection](https://mlsanity.com/paper/2301.01615/) (2023)
- [S4OD: Semi-Supervised learning for Single-Stage Object Detection](https://mlsanity.com/paper/2204.04492/) (2022)
- [Teach-DETR: Better Training DETR with Teachers](https://mlsanity.com/paper/2211.11953/) (2022)
- [A Simple and Generic Framework for Feature Distillation via Channel-wise Transformation](https://mlsanity.com/paper/2303.13212/) (2023)
- [Bridging Cross-task Protocol Inconsistency for Distillation in Dense Object Detection](https://mlsanity.com/paper/2308.14286/) (2023)
- [Object-centric Cross-modal Feature Distillation for Event-based Object Detection](https://mlsanity.com/paper/2311.05494/) (2023)
- [ALSS-YOLO: An Adaptive Lightweight Channel Split and Shuffling Network for TIR Wildlife Detection in UAV Imagery](https://mlsanity.com/paper/2409.06259/) (2024)
- [CLoCKDistill: Consistent Location-and-Context-aware Knowledge Distillation for DETRs](https://mlsanity.com/paper/2502.10683/) (2025)
- [Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models](https://mlsanity.com/paper/2504.13825/) (2025)
- [Task-Balanced Distillation for Object Detection](https://mlsanity.com/paper/2208.03006/) (2022)
- [MutDet: Mutually Optimizing Pre-training for Remote Sensing Object Detection](https://mlsanity.com/paper/2407.09920/) (2024)
- [AMD: Adaptive Masked Distillation for Object Detection](https://mlsanity.com/paper/2301.13538/) (2023)
- [Knowledge Distillation for Object Detection: from generic to remote sensing datasets](https://mlsanity.com/paper/2307.09264/) (2023)
- [RdimKD: Generic Distillation Paradigm by Dimensionality Reduction](https://mlsanity.com/paper/2312.08700/) (2023)

## Abstract (arXiv)

Knowledge distillation (KD) has witnessed its powerful capability in learning compact models in object detection. Previous KD methods for object detection mostly focus on imitating deep features within the imitation regions instead of mimicking classification logit due to its inefficiency in distilling localization information and trivial improvement. In this paper, by reformulating the knowledge distillation process on localization, we present a novel localization distillation (LD) method which can efficiently transfer the localization knowledge from the teacher to the student. Moreover, we also heuristically introduce the concept of valuable localization region that can aid to selectively distill the semantic and localization knowledge for a certain region. Combining these two new components, for the first time, we show that logit mimicking can outperform feature imitation and localization knowledge distillation is more important and efficient than semantic knowledge for distilling object detectors. Our distillation scheme is simple as well as effective and can be easily applied to different dense object detectors. Experiments show that our LD can boost the AP score of GFocal-ResNet-50 with a single-scale 1x training schedule from 40.1 to 42.1 on the COCO benchmark without any sacrifice on the inference speed. Our source code and trained models are publicly available at https://github.com/HikariTJU/LD