mlsanityData to 14 Oct 2026

Humble Teachers Teach Better Students for Semi-Supervised Object Detection

arXiv 2106.10456 ↗First posted 19 Jun 2021cs.CVcited by 116

On the map: Knowledge distillation → Detection and dense prediction distillation

Abstract (arXiv)

We propose a semi-supervised approach for contemporary object detectors following the teacher-student dual model framework. Our method is featured with 1) the exponential moving averaging strategy to update the teacher from the student online, 2) using plenty of region proposals and soft pseudo-labels as the student's training targets, and 3) a light-weighted detection-specific data ensemble for the teacher to generate more reliable pseudo-labels. Compared to the recent state-of-the-art -- STAC, which uses hard labels on sparsely selected hard pseudo samples, the teacher in our model exposes richer information to the student with soft-labels on many proposals. Our model achieves COCO-style AP of 53.04% on VOC07 val set, 8.4% better than STAC, when using VOC12 as unlabeled data. On MS-COCO, it outperforms prior work when only a small percentage of data is taken as labeled. It also reaches 53.8% AP on MS-COCO test-dev with 3.1% gain over the fully supervised ResNet-152 Cascaded R-CNN, by tapping into unlabeled data of a similar size to the labeled data.

Key facts. Cited by 116 papers in the map (data through 14 Oct 2026). Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation. Builds on 41 of its 60 references that are in the map. First posted on arXiv 19 Jun 2021.

Builds on

41 of 60 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  8. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  9. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  10. 2015Fast R-CNNcited by 2,933
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  12. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
29 more
  1. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  2. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  3. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,391
  4. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  5. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  6. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  7. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  8. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,057
  9. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  10. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  11. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  12. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
  13. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 472
  14. 2016Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learningcited by 441
  15. 2020Rethinking Pre-training and Self-trainingcited by 352
  16. 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
  17. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  18. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  19. 2019Label Propagation for Deep Semi-supervised Learningcited by 289
  20. 2020A Simple Semi-Supervised Learning Framework for Object Detectioncited by 282
  21. 2017Integral Human Pose Regressioncited by 256
  22. 2017Data Distillation: Towards Omni-Supervised Learningcited by 126
  23. 2020Proposal Learning for Semi-Supervised Object Detectioncited by 55
  24. 2020Interpolation-based semi-supervised learning for object detectioncited by 41
  25. 2018NOTE-RCNN: NOise Tolerant Ensemble RCNN for Semi-Supervised Object Detectioncited by 37
  26. 2014LSDA: Large Scale Detection Through Adaptationcited by 32
  27. 2018Towards Human-Machine Cooperation: Self-supervised Sample Mining for Object Detectioncited by 32
  28. 2018Visual and Semantic Knowledge Transfer for Large Scale Semi-supervised Object Detectioncited by 22
  29. 2015Improving Object Detection with Deep Convolutional Networks via Bayesian Optimization and Structured Predictioncited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 37; 2023: 31; 2024: 28; 2025: 10; 2026: 3.

116 papers.

Sort:

By area:

  1. 2022Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentationcited by 178
  2. 2022USB: A Unified Semi-supervised Learning Benchmark for Classificationcited by 95
  3. 2022Unbiased Teacher v2: Semi-supervised Object Detection for Anchor-free and Anchor-based Detectorscited by 69
  4. 2022Class-Aware Contrastive Semi-Supervised Learningcited by 62
  5. 2024UniMatch V2: Pushing the Limit of Semi-Supervised Semantic Segmentationcited by 54
  6. 2022Semi-Supervised and Unsupervised Deep Visual Learning: A Surveycited by 52
  7. 2022PseCo: Pseudo Labeling and Consistency Training for Semi-Supervised Object Detectioncited by 51
  8. 2022Dense Teacher: Dense Pseudo-Labels for Semi-supervised Object Detectioncited by 48
  9. 2022Consistent-Teacher: Towards Reducing Inconsistent Pseudo-targets in Semi-supervised Object Detectioncited by 43
  10. 2023Active Teacher for Semi-Supervised Object Detectioncited by 43
  11. 2022Dense Learning based Semi-Supervised Object Detectioncited by 32
  12. 2021Data-Free Knowledge Transfer: A Surveycited by 31
38 more
  1. 2022Omni-DETR: Omni-Supervised Object Detection with Transformerscited by 29
  2. 2022TeST: Test-time Self-Training under Distribution Shiftcited by 29
  3. 2021Semi-Supervised Object Detection with Adaptive Class-Rebalancing Self-Trainingcited by 28
  4. 2024Audio-Visual Segmentation via Unlabeled Frame Exploitationcited by 28
  5. 2021Not All Labels Are Equal: Rationalizing The Labeling Costs for Training Object Detectioncited by 27
  6. 2023Hierarchical Supervision and Shuffle Data Augmentation for 3D Semi-Supervised Object Detectioncited by 24
  7. 2022Mix-Teaching: A Simple, Unified and Effective Semi-Supervised Learning Framework for Monocular 3D Object Detectioncited by 23
  8. 2022Noisy Boundaries: Lemon or Lemonade for Semi-supervised Instance Segmentation?cited by 21
  9. 2022DetMatch: Two Teachers are Better Than One for Joint 2D and 3D Semi-Supervised Object Detectioncited by 20
  10. 2023SOOD: Towards Semi-Supervised Oriented Object Detectioncited by 19
  11. 2021Combating Noise: Semi-supervised Learning by Region Uncertainty Quantificationcited by 18
  12. 2023DDS3D: Dense Pseudo-Labels with Dynamic Threshold for Semi-Supervised 3D Object Detectioncited by 18
  13. 2023Doubly Robust Self-Trainingcited by 18
  14. 2022End-to-End Semi-Supervised Learning for Video Action Detectioncited by 17
  15. 2022Open-Set Semi-Supervised Object Detectioncited by 17
  16. 2023Efficient Teacher: Semi-Supervised Object Detection for YOLOv5cited by 17
  17. 2023Pseudo-label Alignment for Semi-supervised Instance Segmentationcited by 17
  18. 2022Label-Efficient Online Continual Object Detection in Streaming Videocited by 16
  19. 2023DQS3D: Densely-matched Quantization-aware Semi-supervised 3D Detectioncited by 16
  20. 2022Scale-Equivalent Distillation for Semi-Supervised Object Detectioncited by 15
  21. 2023Ambiguity-Resistant Semi-Supervised Learning for Dense Object Detectioncited by 14
  22. 2023MixTeacher: Mining Promising Labels with Mixed Scale Teacher for Semi-Supervised Object Detectioncited by 13
  23. 2024Sparse Semi-DETR: Sparse Learnable Queries for Semi-Supervised Object Detectioncited by 13
  24. 2024Crowd-SAM: SAM as a Smart Annotator for Object Detection in Crowded Scenescited by 13
  25. 2022DTG-SSOD: Dense Teacher Guidance for Semi-Supervised Object Detectioncited by 12
  26. 2024SOOD++: Leveraging Unlabeled Data to Boost Oriented Object Detectioncited by 12
  27. 2025CGMatch: A Different Perspective of Semi-supervised Learningcited by 11
  28. 2022PseudoProp: Robust Pseudo-Label Generation for Semi-Supervised Object Detection in Autonomous Driving Systemscited by 10
  29. 2022Dynamic Curriculum Learning for Great Ape Detection in the Wildcited by 10
  30. 2023Virtual Category Learning: A Semi-Supervised Learning Method for Dense Prediction with Extremely Limited Labelscited by 10
  31. 2023Pseudo Flow Consistency for Self-Supervised 6D Object Pose Estimationcited by 9
  32. 2024Dual-Perspective Knowledge Enrichment for Semi-Supervised 3D Object Detectioncited by 9
  33. 2023ALWOD: Active Learning for Weakly-Supervised Object Detectioncited by 8
  34. 2021Self-supervised Learning for Semi-supervised Temporal Language Groundingcited by 7
  35. 2021Probabilistic Contrastive Learning for Domain Adaptationcited by 7
  36. 2023Towards End-to-End Semi-Supervised Table Detection with Deformable Transformercited by 7
  37. 2024ACTRESS: Active Retraining for Semi-supervised Visual Groundingcited by 7
  38. 2022CrossRectify: Leveraging Disagreement for Semi-supervised Object Detectioncited by 6

50 most cited of 116 All 116 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Label Matching Semi-Supervised Object Detection
  2. 2024Co-Learning: Towards Semi-Supervised Object Detection with Road-side Cameras
  3. 2021Cross-Domain Adaptive Teacher for Object Detection
  4. 2024Versatile Teacher: A Class-aware Teacher-student Framework for Cross-domain Adaptation
  5. 2021Revisiting Knowledge Distillation for Object Detection
  6. 2022Boosting Semi-Supervised 3D Object Detection with Semi-Sampling
  7. 2024Semi-Supervised 3D Object Detection with Channel Augmentation using Transformation Equivariance
  8. 2023Switching Temporary Teachers for Semi-Supervised Semantic Segmentation