# Scale-Equivalent Distillation for Semi-Supervised Object Detection

arXiv: https://arxiv.org/abs/2203.12244 · HTML: https://mlsanity.com/paper/2203.12244/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 50 references that are in the map.
- First posted on arXiv 23 Mar 2022.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Rethinking ImageNet Pre-training](https://mlsanity.com/paper/1811.08883/) (2018)
- [Rethinking Pre-training and Self-training](https://mlsanity.com/paper/2006.06882/) (2020)
- [Unbiased Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2102.09480/) (2021)
- [A Simple Semi-Supervised Learning Framework for Object Detection](https://mlsanity.com/paper/2005.04757/) (2020)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Data Distillation: Towards Omni-Supervised Learning](https://mlsanity.com/paper/1712.04440/) (2017)
- [An Analysis of Scale Invariance in Object Detection - SNIP](https://mlsanity.com/paper/1711.08189/) (2017)
- [Humble Teachers Teach Better Students for Semi-Supervised Object Detection](https://mlsanity.com/paper/2106.10456/) (2021)

## Built on by (most cited first, 15 of 15)

- [Semi-DETR: Semi-Supervised Object Detection with Detection Transformers](https://mlsanity.com/paper/2307.08095/) (2023)
- [MixTeacher: Mining Promising Labels with Mixed Scale Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2303.09061/) (2023)
- [Sparse Semi-DETR: Sparse Learnable Queries for Semi-Supervised Object Detection](https://mlsanity.com/paper/2404.01819/) (2024)
- [DTG-SSOD: Dense Teacher Guidance for Semi-Supervised Object Detection](https://mlsanity.com/paper/2207.05536/) (2022)
- [Semi-Supervised Object Detection with Object-wise Contrastive Learning and Regression Uncertainty](https://mlsanity.com/paper/2212.02747/) (2022)
- [Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformer](https://mlsanity.com/paper/2407.08460/) (2024)
- [Gradient-based Sampling for Class Imbalanced Semi-supervised Object Detection](https://mlsanity.com/paper/2403.15127/) (2024)
- [A Decade of You Only Look Once (YOLO) for Object Detection: A Review](https://mlsanity.com/paper/2504.18586/) (2025)
- [Semi-Supervised and Long-Tailed Object Detection with CascadeMatch](https://mlsanity.com/paper/2305.14813/) (2023)
- [Training-based Model Refinement and Representation Disagreement for Semi-Supervised Object Detection](https://mlsanity.com/paper/2307.13755/) (2023)
- [Low-Confidence Samples Mining for Semi-supervised Object Detection](https://mlsanity.com/paper/2306.16201/) (2023)
- [Density Crop-guided Semi-supervised Object Detection in Aerial Images](https://mlsanity.com/paper/2308.05032/) (2023)
- [Aligning Data Selection with Performance: Performance-driven Reinforcement Learning for Active Learning in Object Detection](https://mlsanity.com/paper/2310.08387/) (2023)
- [Rethinking Scale Imbalance in Semi-supervised Object Detection for Aerial Images](https://mlsanity.com/paper/2310.14718/) (2023)
- [Collaboration of Teachers for Semi-supervised Object Detection](https://mlsanity.com/paper/2405.13374/) (2024)

## Abstract (arXiv)

Recent Semi-Supervised Object Detection (SS-OD) methods are mainly based on self-training, i.e., generating hard pseudo-labels by a teacher model on unlabeled data as supervisory signals. Although they achieved certain success, the limited labeled data in semi-supervised learning scales up the challenges of object detection. We analyze the challenges these methods meet with the empirical experiment results. We find that the massive False Negative samples and inferior localization precision lack consideration. Besides, the large variance of object sizes and class imbalance (i.e., the extreme ratio between background and object) hinder the performance of prior arts. Further, we overcome these challenges by introducing a novel approach, Scale-Equivalent Distillation (SED), which is a simple yet effective end-to-end knowledge distillation framework robust to large object size variance and class imbalance. SED has several appealing benefits compared to the previous works. (1) SED imposes a consistency regularization to handle the large scale variance problem. (2) SED alleviates the noise problem from the False Negative samples and inferior localization precision. (3) A re-weighting strategy can implicitly screen the potential foreground regions of the unlabeled data to reduce the effect of class imbalance. Extensive experiments show that SED consistently outperforms the recent state-of-the-art methods on different datasets with significant margins. For example, it surpasses the supervised counterpart by more than 10 mAP when using 5% and 10% labeled data on MS-COCO.