mlsanityData to 7 Oct 2026

General Instance Distillation for Object Detection

arXiv 2103.02340 ↗First posted 3 Mar 2021cs.CVcited by 113

Not on the map. Why?

Abstract (arXiv)

In recent years, knowledge distillation has been proved to be an effective solution for model compression. This approach can make lightweight student models acquire the knowledge extracted from cumbersome teacher models. However, previous distillation methods of detection have weak generalization for different detection frameworks and rely heavily on ground truth (GT), ignoring the valuable relation information between instances. Thus, we propose a novel distillation method for detection tasks based on discriminative instances without considering the positive or negative distinguished by GT, which is called general instance distillation (GID). Our approach contains a general instance selection module (GISM) to make full use of feature-based, relation-based and response-based knowledge for distillation. Extensive results demonstrate that the student model achieves significant AP improvement and even outperforms the teacher in various detection frameworks. Specifically, RetinaNet with ResNet-50 achieves 39.1% in mAP with GID on COCO dataset, which surpasses the baseline 36.2% by 2.9%, and even better than the ResNet-101 based teacher model with 38.1% AP.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 3 Mar 2021.

Builds on

26 of 37 references.

References · 26

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2017Mask R-CNNcited by 7,259
  5. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  7. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  8. 2018YOLOv3: An Incremental Improvementcited by 2,652
  9. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  10. 2020Knowledge Distillation: A Surveycited by 1,578
  11. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  12. 2016YOLO9000: Better, Faster, Strongercited by 1,429
14 more
  1. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  2. 2019Relational Knowledge Distillationcited by 873
  3. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  4. 2019Contrastive Representation Distillationcited by 734
  5. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  6. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  7. 2019Similarity-Preserving Knowledge Distillationcited by 507
  8. 2019Variational Information Distillation for Knowledge Transfercited by 321
  9. 2017Relation Networks for Object Detectioncited by 305
  10. 2018Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neuronscited by 235
  11. 2019Distilling Object Detectors with Fine-grained Feature Imitationcited by 188
  12. 2020Explaining Knowledge Distillation by Quantifying the Knowledgecited by 65
  13. 2020Distilling Object Detectors with Task Adaptive Regularizationcited by 36
  14. 2020Subclass Distillationcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 35; 2023: 35; 2024: 20; 2025: 7; 2026: 4.

113 papers.

Sort:

By area:

  1. 2021Focal and Global Knowledge Distillation for Detectorscited by 138
  2. 2022Masked Generative Distillationcited by 107
  3. 2024D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinementcited by 73
  4. 2022Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillationcited by 72
  5. 2022Knowledge Distillation via the Target-aware Transformercited by 70
  6. 2022MonoDistill: Learning Spatial Features for Monocular 3D Object Detectioncited by 68
  7. 2022BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detectioncited by 66
  8. 2023Object-Aware Distillation Pyramid for Open-Vocabulary Object Detectioncited by 63
  9. 2023CrossKD: Cross-Head Knowledge Distillation for Object Detectioncited by 50
  10. 2022Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detectioncited by 48
  11. 2021ViDT: An Efficient and Effective Fully Transformer-based Object Detectorcited by 45
  12. 2023UniDistill: A Universal Cross-Modality Knowledge Distillation Framework for 3D Object Detection in Bird's-Eye Viewcited by 43
38 more
  1. 2021Instance-Conditional Knowledge Distillation for Object Detectioncited by 42
  2. 2022Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillationcited by 41
  3. 2022PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficientcited by 41
  4. 2023DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillationcited by 41
  5. 2021Knowledge Distillation for Object Detection via Rank Mimicking and Prediction-guided Feature Imitationcited by 40
  6. 2022Cross-Domain Correlation Distillation for Unsupervised Domain Adaptation in Nighttime Semantic Segmentationcited by 40
  7. 2021Distilling Object Detectors with Feature Richnesscited by 38
  8. 2022TiG-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learningcited by 37
  9. 2022DETRDistill: A Universal Knowledge Distillation Framework for DETR-familiescited by 36
  10. 2022PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detectioncited by 33
  11. 2022Towards Efficient 3D Object Detection with Knowledge Distillationcited by 29
  12. 2023X3KD: Knowledge Distillation Across Modalities, Tasks and Stages for Multi-Camera 3D Object Detectioncited by 28
  13. 2023FreeKD: Knowledge Distillation via Semantic Frequency Promptcited by 27
  14. 2023RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillationcited by 27
  15. 2023A Machine Learning-oriented Survey on Tiny Machine Learningcited by 26
  16. 2022Masked Distillation with Receptive Tokenscited by 23
  17. 2022Boosting 3D Object Detection by Simulating Multimodality on Point Cloudscited by 22
  18. 2023Function-Consistent Feature Distillationcited by 21
  19. 2021LGD: Label-guided Self-distillation for Object Detectioncited by 19
  20. 2022Localization Distillation for Object Detectioncited by 19
  21. 2025ACAM-KD: Adaptive and Cooperative Attention Masking for Knowledge Distillationcited by 18
  22. 2021Improving Object Detection by Label Assignment Distillationcited by 17
  23. 2022D3ETR: Decoder Distillation for Detection Transformercited by 17
  24. 2024Rethinking Centered Kernel Alignment in Knowledge Distillationcited by 17
  25. 2022Prediction-Guided Distillation for Dense Object Detectioncited by 14
  26. 2022Structural Knowledge Distillation for Object Detectioncited by 14
  27. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  28. 2023SupFusion: Supervised LiDAR-Camera Fusion for 3D Object Detectioncited by 14
  29. 2022HEAD: HEtero-Assists Distillation for Heterogeneous Object Detectorscited by 12
  30. 2022KD-DETR: Knowledge Distillation for Detection Transformer with Consistent Distillation Points Samplingcited by 12
  31. 2022Guided Hybrid Quantization for Object detection in Multimodal Remote Sensing Imagery via One-to-one Self-teachingcited by 12
  32. 2023DAMO-StreamNet: Optimizing Streaming Perception in Autonomous Drivingcited by 12
  33. 2023Decomposed Cross-modal Distillation for RGB-based Temporal Action Detectioncited by 12
  34. 2023Gradient-Guided Knowledge Distillation for Object Detectorscited by 11
  35. 2022Delta Distillation for Efficient Video Processingcited by 10
  36. 2023Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillationcited by 10
  37. 2023Spatial Self-Distillation for Object Detection with Inaccurate Bounding Boxescited by 9
  38. 2024LabelDistill: Label-guided Cross-modal Knowledge Distillation for Camera-based 3D Object Detectioncited by 9

50 most cited of 113 All 113 (JSON)