mlsanityData to 14 Oct 2026

Learnable Instance Attention Filtering for Adaptive Detector Distillation

arXiv 2603.26088 ↗First posted 27 Mar 2026cs.CVcited by 0

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

As deep vision models grow increasingly complex to achieve higher performance, deployment efficiency has become a critical concern. Knowledge distillation (KD) mitigates this issue by transferring knowledge from large teacher models to compact student models. While many feature-based KD methods rely on spatial filtering to guide distillation, they typically treat all object instances uniformly, ignoring instance-level variability. Moreover, existing attention filtering mechanisms are typically heuristic or teacher-driven, rather than learned with the student. To address these limitations, we propose Learnable Instance Attention Filtering for Adaptive Detector Distillation (LIAF-KD), a novel framework that introduces learnable instance selectors to dynamically evaluate and reweight instance importance during distillation. Notably, the student contributes to this process based on its evolving learning state. Experiments on the KITTI and COCO datasets demonstrate consistent improvements, with a 2% gain on a GFL ResNet-50 student without added complexity, outperforming state-of-the-art methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 34 of its 41 references that are in the map. First posted on arXiv 27 Mar 2026.

Builds on

34 of 41 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Gradient-Guided Knowledge Distillation for Object Detectors7 Mar
  2. 2022Adaptive Instance Distillation for Object Detection in Autonomous Driving26 Jan
  3. 2021Distilling Object Detectors via Decoupled Features26 Mar
  4. 2021General Instance Distillation for Object Detection3 Mar

Other references · 30

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2017Mask R-CNNcited by 7,277
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  8. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  9. 2015Fast R-CNNcited by 2,933
  10. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  11. 2020Knowledge Distillation: A Surveycited by 1,603
  12. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
18 more
  1. 2021Conditional DETR for Fast Training Convergencecited by 424
  2. 2020Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detectioncited by 409
  3. 2019Distilling Object Detectors with Fine-grained Feature Imitationcited by 189
  4. 2023A Conditional Denoising Diffusion Probabilistic Model for Point Cloud Upsamplingcited by 28
  5. 2023FreeKD: Knowledge Distillation via Semantic Frequency Promptcited by 27
  6. 2024Unsupervised Hyperspectral Image Super-Resolution via Self-Supervised Modality Decouplingcited by 26
  7. 2026EvoTool: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selectioncited by 24
  8. 2022Masked Distillation with Receptive Tokenscited by 23
  9. 2024An End-to-End Robust Point Cloud Semantic Segmentation Network with Single-Step Conditional Diffusion Modelscited by 23
  10. 2026ToolTree: Efficient LLM Agent Tool Planning via Dual-Feedback Monte Carlo Tree Search and Bidirectional Pruningcited by 19
  11. 2026Align then Adapt: Rethinking Parameter-Efficient Transfer Learning in 4D Perceptioncited by 18
  12. 2024Generative Regression Based Watch Time Prediction for Short-Video Recommendationcited by 13
  13. 2025Robust Single-Stage Fully Sparse 3D Object Detection via Detachable Latent Diffusioncited by 12
  14. 2025PointDico: Contrastive 3D Representation Learning Guided by Diffusion Modelscited by 11
  15. 2025MS-DETR: Towards Effective Video Moment Retrieval and Highlight Detection by Joint Motion-Semantic Learningcited by 10
  16. 2026CMHANet: A Cross-Modal Hybrid Attention Network for Point Cloud Registrationcited by 9
  17. 2025Fine-Grained Zero-Shot Object Detectioncited by 8
  18. 2026TDEC: Deep Embedded Image Clustering with Transformer and Distribution Informationcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ACAM-KD: Adaptive and Cooperative Attention Masking for Knowledge Distillation
  2. 2023Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation
  3. 2025Architectural Insights into Knowledge Distillation for Object Detection: A Comprehensive Review
  4. 2023Efficient Object Detection in Optical Remote Sensing Imagery via Attention-based Feature Distillation
  5. 2021Distilling Image Classifiers in Object Detectors
  6. 2021Instance-Conditional Knowledge Distillation for Object Detection
  7. 2024Task Integration Distillation for Object Detectors
  8. 2022Prediction-Guided Distillation for Dense Object Detection