# SAMKD: Spatial-aware Adaptive Masking Knowledge Distillation for Object Detection

arXiv: https://arxiv.org/abs/2501.07101 · HTML: https://mlsanity.com/paper/2501.07101/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 24 of its 30 references that are in the map.
- First posted on arXiv 13 Jan 2025.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 24 of 24)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [YOLOv10: Real-Time End-to-End Object Detection](https://mlsanity.com/paper/2405.14458/) (2024)
- [Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection](https://mlsanity.com/paper/2006.04388/) (2020)
- [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022)
- [Distilling Knowledge via Knowledge Review](https://mlsanity.com/paper/2104.09044/) (2021)
- [Oriented R-CNN for Object Detection](https://mlsanity.com/paper/2108.05699/) (2021)
- [MambaVision: A Hybrid Mamba-Transformer Vision Backbone](https://mlsanity.com/paper/2407.08083/) (2024)
- [Focal and Global Knowledge Distillation for Detectors](https://mlsanity.com/paper/2111.11837/) (2021)
- [Masked Generative Distillation](https://mlsanity.com/paper/2205.01529/) (2022)
- [Localization Distillation for Dense Object Detection](https://mlsanity.com/paper/2102.12252/) (2021)
- [PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient](https://mlsanity.com/paper/2207.02039/) (2022)
- [FreeKD: Knowledge Distillation via Semantic Frequency Prompt](https://mlsanity.com/paper/2311.12079/) (2023)
- [Scale Decoupled Distillation](https://mlsanity.com/paper/2403.13512/) (2024)
- [AMD: Adaptive Masked Distillation for Object Detection](https://mlsanity.com/paper/2301.13538/) (2023)
- [DMKD: Improving Feature-based Knowledge Distillation for Object Detection Via Dual Masking Augmentation](https://mlsanity.com/paper/2309.02719/) (2023)
- [Tree-structured Auxiliary Online Knowledge Distillation](https://mlsanity.com/paper/2208.10068/) (2022)

## Built on by (most cited first, 2 of 2)

- [Progressive Class-level Distillation](https://mlsanity.com/paper/2505.24310/) (2025)
- [Lesion Detection in CT with Frozen Self-Distilled Features: SALT, a Spatially Adaptive Label-Guided Temperature](https://mlsanity.com/paper/2608.05100/) (2026)

## Abstract (arXiv)

Most of recent attention-guided feature masking distillation methods perform knowledge transfer via global teacher attention maps without delving into fine-grained clues. Instead, performing distillation at finer granularity is conducive to uncovering local details supplementary to global knowledge transfer and reconstructing comprehensive student features. In this study, we propose a Spatial-aware Adaptive Masking Knowledge Distillation (SAMKD) framework for accurate object detection. Different from previous feature distillation methods which mainly perform single-scale feature masking, we develop spatially hierarchical feature masking distillation scheme, such that the object-aware locality is encoded during coarse-to-fine distillation process for improved feature reconstruction. In addition, our spatial-aware feature distillation strategy is combined with a masking logit distillation scheme in which region-specific feature difference between teacher and student networks is utilized to adaptively guide the distillation process. Thus, it can help the student model to better learn from the teacher counterpart with improved knowledge transfer and reduced gap. Extensive experiments for detection task demonstrate the superiority of our method. For example, when FCOS is used as teacher detector with ResNet101 backbone, our method improves the student network from 35.3% to 38.8% mAP, outperforming state-of-the-art distillation methods including MGD, FreeKD and DMKD.