mlsanityData to 14 Oct 2026

SEA: Bridging the Gap Between One- and Two-stage Detector Distillation via SEmantic-aware Alignment

arXiv 2203.00862 ↗First posted 2 Mar 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We revisit the one- and two-stage detector distillation tasks and present a simple and efficient semantic-aware framework to fill the gap between them. We address the pixel-level imbalance problem by designing the category anchor to produce a representative pattern for each category and regularize the topological distance between pixels and category anchors to further tighten their semantic bonds. We name our method SEA (SEmantic-aware Alignment) distillation given the nature of abstracting dense fine-grained information by semantic reliance to well facilitate distillation efficacy. SEA is well adapted to either detection pipeline and achieves new state-of-the-art results on the challenging COCO object detection task on both one- and two-stage detectors. Its superior performance on instance segmentation further manifests the generalization ability. Both 2x-distilled RetinaNet and FCOS with ResNet50-FPN outperform their corresponding 3x ResNet101-FPN teacher, arriving 40.64 and 43.06 AP, respectively. Code will be made publicly available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 46 references that are in the map. First posted on arXiv 2 Mar 2022.

Builds on

39 of 46 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2017Mask R-CNNcited by 7,277
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  10. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  11. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  12. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
27 more
  1. 2015Fast R-CNNcited by 2,933
  2. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  3. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  4. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  5. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  7. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  8. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  9. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  10. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  11. 2017Focal Loss for Dense Object Detectioncited by 752
  12. 2017Deep Mutual Learningcited by 628
  13. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  14. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  15. 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 540
  16. 2019Hybrid Task Cascade for Instance Segmentationcited by 481
  17. 2019Cascade R-CNN: High Quality Object Detection and Instance Segmentationcited by 429
  18. 2020Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detectioncited by 409
  19. 2019Structured Knowledge Distillation for Dense Predictioncited by 279
  20. 2019Distilling Object Detectors with Fine-grained Feature Imitationcited by 189
  21. 2020Channel-wise Knowledge Distillation for Dense Predictioncited by 163
  22. 2021You Only Look One-level Featurecited by 116
  23. 2021General Instance Distillation for Object Detectioncited by 114
  24. 2021Distilling Object Detectors via Decoupled Featurescited by 106
  25. 2020Dynamic R-CNN: Towards High Quality Object Detection via Dynamic Trainingcited by 103
  26. 2019Rethinking Classification and Localization for Object Detectioncited by 98
  27. 2021Deep Structured Instance Graph for Distilling Object Detectorscited by 9

Built on by

Not cited yet.