mlsanityData to 14 Oct 2026

Detection and dense prediction distillation

451 papers0 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: distillation, detection, knowledge, object, knowledge distillation

Key facts. Detection and dense prediction distillation: 451 papers on arXiv since 2021 (data through 14 Oct 2026). 54 of its 451 mapped papers (12%) cite “General Instance Distillation for Object Detection” (2021, arXiv 2103.02340). 6 new papers 25 Sep–14 Oct.

What’s new

25 Sep – 14 Oct 2026

6 new papers, 25 Sep – 14 Oct; 0 this week.

  1. Earlier · 6
  2. One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
  3. Difference Feature Map Distillation: Transferring Inter-Sample Relational Knowledge Towards Efficient Transformer-Based Tracking
  4. Shadow Feature Refinement Network: Progressive Feature Refinement based on Knowledge Distillation for Effective Shadow Removal
  5. WinoTS: Wavelet-based Self-Distillation for Time Series Models also fits: General knowledge distillation methods
  6. Emergent Multi-View Geometry Through Self-Distillation
  7. SFE-VGGT: Source-Free VGGT Distillation for Event-Based Monocular Depth Estimation

Foundations

Most cited by papers here

  1. General Instance Distillation for Object Detection2021cited by 54 here
  2. Focal and Global Knowledge Distillation for Detectors2021cited by 54 here
  3. Distilling Object Detectors via Decoupled Features2021cited by 52 here
  4. Localization Distillation for Dense Object Detection2021cited by 28 here
  5. Instance-Conditional Knowledge Distillation for Object Detection2021cited by 24 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2021General Instance Distillation for Object Detectioncited by 54 here
  2. 2021Instance-Conditional Knowledge Distillation for Object Detectioncited by 24 here
  3. 2021Focal and Global Knowledge Distillation for Detectorscited by 54 here
  4. 2022Masked Generative Distillationcited by 34 here
  5. 2022BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detectioncited by 21 here
  6. 2023DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillationcited by 15 here
18 more it builds on, not in this chain
  1. 2021Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspectivecited by 12 here
  2. 2021Localization Distillation for Dense Object Detectioncited by 28 here
  3. 2021Distilling Object Detectors via Decoupled Featurescited by 52 here
  4. 2021Distilling Knowledge via Knowledge Reviewcited by 27 here
  5. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 11 here
  6. 2021Knowledge Distillation for Object Detection via Rank Mimicking and Prediction-guided Feature Imitationcited by 14 here
  7. 2022Anomaly Detection via Reverse Distillation from One-Class Embeddingcited by 18 here
  8. 2022Decoupled Knowledge Distillationcited by 36 here
  9. 2022Cross-Image Relational Knowledge Distillation for Semantic Segmentationcited by 24 here
  10. 2022Knowledge Distillation from A Stronger Teachercited by 15 here
  11. 2022PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detectioncited by 15 here
  12. 2022Towards Efficient 3D Object Detection with Knowledge Distillationcited by 18 here
  13. 2022Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentationcited by 13 here
  14. 2022PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficientcited by 18 here
  15. 2022Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detectioncited by 14 here
  16. 2023UniDistill: A Universal Cross-Modality Knowledge Distillation Framework for 3D Object Detection in Bird's-Eye Viewcited by 16 here
  17. 2023CrossKD: Cross-Head Knowledge Distillation for Object Detectioncited by 14 here
  18. 2023FreeKD: Knowledge Distillation via Semantic Frequency Promptcited by 11 here

The 24 most cited of 352.

Browse

30 of 451, ranked 6–35

  1. 6Cross-Image Relational Knowledge Distillation for Semantic Segmentation202224
  2. 7BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detection202221
  3. 8Anomaly Detection via Reverse Distillation from One-Class Embedding202218
  4. 9Towards Efficient 3D Object Detection with Knowledge Distillation202218
  5. 10PKD: General Distillation Framework for Object Detectors via Pearson Correlation Coefficient202218
  6. 11UniDistill: A Universal Cross-Modality Knowledge Distillation Framework for 3D Object Detection in Bird's-Eye View202316
  7. 12DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillation202315
  8. 13PointDistiller: Structured Knowledge Distillation Towards Efficient and Compact 3D Detection202215
  9. 14CrossKD: Cross-Head Knowledge Distillation for Object Detection202314
  10. 15Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection202214
All 30
  1. 16Knowledge Distillation for Object Detection via Rank Mimicking and Prediction-guided Feature Imitation202114
  2. 17Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation202213
  3. 18Open-vocabulary Object Detection via Vision and Language Knowledge Distillation202111
  4. 19G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation202110
  5. 20Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation20239
  6. 21Prediction-Guided Distillation for Dense Object Detection20228
  7. 22DETRDistill: A Universal Knowledge Distillation Framework for DETR-families20228
  8. 23X3KD: Knowledge Distillation Across Modalities, Tasks and Stages for Multi-Camera 3D Object Detection20237
  9. 24itKD: Interchange Transfer-based Knowledge Distillation for 3D Object Detection20227
  10. 25Online Knowledge Distillation for Efficient Pose Estimation20217
  11. 26LGD: Label-guided Self-distillation for Object Detection20217
  12. 27Cross-Domain Adaptive Teacher for Object Detection20217
  13. 28RadarDistill: Boosting Radar-based Object Detection Performance via Knowledge Distillation from LiDAR Features20246
  14. 29CRKD: Enhanced Camera-Radar Object Detection with Cross-modality Knowledge Distillation20246
  15. 30BPKD: Boundary Privileged Knowledge Distillation For Semantic Segmentation20236
  16. 31Augmentation-Free Dense Contrastive Knowledge Distillation for Efficient Semantic Segmentation20236
  17. 32TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation20226
  18. 33Localization Distillation for Object Detection20226
  19. 34Structural Knowledge Distillation for Object Detection20226
  20. 35Humble Teachers Teach Better Students for Semi-Supervised Object Detection20216