# Localization Distillation for Object Detection

arXiv: https://arxiv.org/abs/2204.05957 · HTML: https://mlsanity.com/paper/2204.05957/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 92 of its 118 references that are in the map.
- First posted on arXiv 12 Apr 2022.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 30 of 92)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Improved Knowledge Distillation via Teacher Assistant](https://mlsanity.com/paper/1902.03393/) (2019)

## Built on by (most cited first, 19 of 19)

- [Large Selective Kernel Network for Remote Sensing Object Detection](https://mlsanity.com/paper/2303.09030/) (2023)
- [YOLOv6 v3.0: A Full-Scale Reloading](https://mlsanity.com/paper/2301.05586/) (2023)
- [MixFormerV2: Efficient Fully Transformer Tracking](https://mlsanity.com/paper/2305.15896/) (2023)
- [LSKNet: A Foundation Lightweight Backbone for Remote Sensing](https://mlsanity.com/paper/2403.11735/) (2024)
- [MapDistill: Boosting Efficient Camera-based HD Map Construction via Camera-LiDAR Fusion Model Distillation](https://mlsanity.com/paper/2407.11682/) (2024)
- [MedDet: Generative Adversarial Distillation for Efficient Cervical Disc Herniation Detection](https://mlsanity.com/paper/2409.00204/) (2024)
- [Efficient Object Detection in Optical Remote Sensing Imagery via Attention-based Feature Distillation](https://mlsanity.com/paper/2310.18676/) (2023)
- [Distilling Object Detectors With Global Knowledge](https://mlsanity.com/paper/2210.09022/) (2022)
- [Knowledge-guided Causal Intervention for Weakly-supervised Object Localization](https://mlsanity.com/paper/2301.01060/) (2023)
- [Local Dense Logit Relations for Enhanced Knowledge Distillation](https://mlsanity.com/paper/2507.15911/) (2025)
- [LumiNet: Perception-Driven Knowledge Distillation via Statistical Logit Calibration](https://mlsanity.com/paper/2310.03669/) (2023)
- [Explore Spatio-temporal Aggregation for Insubstantial Object Detection: Benchmark Dataset and Baseline](https://mlsanity.com/paper/2206.11459/) (2022)
- [Few-shot Object Localization](https://mlsanity.com/paper/2403.12466/) (2024)
- [Teaching with Uncertainty: Unleashing the Potential of Knowledge Distillation in Object Detection](https://mlsanity.com/paper/2406.06999/) (2024)
- [Learning Using Privileged Information for Litter Detection](https://mlsanity.com/paper/2508.04124/) (2025)
- [Dual-Stream Spectral Decoupling Distillation for Remote Sensing Object Detection](https://mlsanity.com/paper/2512.04413/) (2025)
- [Enhancing Object Detection with Privileged Information: A Model-Agnostic Teacher-Student Approach](https://mlsanity.com/paper/2601.02016/) (2026)
- [Exploring Graph-based Knowledge: Multi-Level Feature Distillation via Channels Relational Graph](https://mlsanity.com/paper/2405.08547/) (2024)
- [Integrating Knowledge Distillation Methods: A Sequential Multi-Stage Framework](https://mlsanity.com/paper/2601.15657/) (2026)

## Abstract (arXiv)

Previous knowledge distillation (KD) methods for object detection mostly focus on feature imitation instead of mimicking the prediction logits due to its inefficiency in distilling the localization information. In this paper, we investigate whether logit mimicking always lags behind feature imitation. Towards this goal, we first present a novel localization distillation (LD) method which can efficiently transfer the localization knowledge from the teacher to the student. Second, we introduce the concept of valuable localization region that can aid to selectively distill the classification and localization knowledge for a certain region. Combining these two new components, for the first time, we show that logit mimicking can outperform feature imitation and the absence of localization distillation is a critical reason for why logit mimicking underperforms for years. The thorough studies exhibit the great potential of logit mimicking that can significantly alleviate the localization ambiguity, learn robust feature representation, and ease the training difficulty in the early stage. We also provide the theoretical connection between the proposed LD and the classification KD, that they share the equivalent optimization effect. Our distillation scheme is simple as well as effective and can be easily applied to both dense horizontal object detectors and rotated object detectors. Extensive experiments on the MS COCO, PASCAL VOC, and DOTA benchmarks demonstrate that our method can achieve considerable AP improvement without any sacrifice on the inference speed. Our source code and pretrained models are publicly available at https://github.com/HikariTJU/LD.