# Training-based Model Refinement and Representation Disagreement for Semi-Supervised Object Detection

arXiv: https://arxiv.org/abs/2307.13755 · HTML: https://mlsanity.com/paper/2307.13755/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 26 of 26)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [How does Disagreement Help Generalization against Label Corruption?](https://mlsanity.com/paper/1901.04215/) (2019)
- [End-to-End Semi-Supervised Object Detection with Soft Teacher](https://mlsanity.com/paper/2106.09018/) (2021)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [Unbiased Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2102.09480/) (2021)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [A Simple Semi-Supervised Learning Framework for Object Detection](https://mlsanity.com/paper/2005.04757/) (2020)
- [Decoupling "when to update" from "how to update"](https://mlsanity.com/paper/1706.02613/) (2017)
- [Humble Teachers Teach Better Students for Semi-Supervised Object Detection](https://mlsanity.com/paper/2106.10456/) (2021)
- [Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework](https://mlsanity.com/paper/2103.11402/) (2021)
- [Unbiased Teacher v2: Semi-supervised Object Detection for Anchor-free and Anchor-based Detectors](https://mlsanity.com/paper/2206.09500/) (2022)
- [Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods](https://mlsanity.com/paper/1903.11260/) (2019)
- [PseCo: Pseudo Labeling and Consistency Training for Semi-Supervised Object Detection](https://mlsanity.com/paper/2203.16317/) (2022)
- [Rethinking Pseudo Labels for Semi-Supervised Object Detection](https://mlsanity.com/paper/2106.00168/) (2021)
- [Label Matching Semi-Supervised Object Detection](https://mlsanity.com/paper/2206.06608/) (2022)
- [Active Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2303.08348/) (2023)
- [Semi-Supervised Object Detection with Adaptive Class-Rebalancing Self-Training](https://mlsanity.com/paper/2107.05031/) (2021)
- [MUM : Mix Image Tiles and UnMix Feature Tiles for Semi-Supervised Object Detection](https://mlsanity.com/paper/2111.10958/) (2021)
- [Scale-Equivalent Distillation for Semi-Supervised Object Detection](https://mlsanity.com/paper/2203.12244/) (2022)
- [Semi-supervised Object Detection via Virtual Category Learning](https://mlsanity.com/paper/2207.03433/) (2022)
- [Mind the Gap: Polishing Pseudo labels for Accurate Semi-supervised Object Detection](https://mlsanity.com/paper/2207.08185/) (2022)

## Built on by (most cited first, 1 of 1)

- [SOOD++: Leveraging Unlabeled Data to Boost Oriented Object Detection](https://mlsanity.com/paper/2407.01016/) (2024)

## Abstract (arXiv)

Semi-supervised object detection (SSOD) aims to improve the performance and generalization of existing object detectors by utilizing limited labeled data and extensive unlabeled data. Despite many advances, recent SSOD methods are still challenged by inadequate model refinement using the classical exponential moving average (EMA) strategy, the consensus of Teacher-Student models in the latter stages of training (i.e., losing their distinctiveness), and noisy/misleading pseudo-labels. This paper proposes a novel training-based model refinement (TMR) stage and a simple yet effective representation disagreement (RD) strategy to address the limitations of classical EMA and the consensus problem. The TMR stage of Teacher-Student models optimizes the lightweight scaling operation to refine the model's weights and prevent overfitting or forgetting learned patterns from unlabeled data. Meanwhile, the RD strategy helps keep these models diverged to encourage the student model to explore additional patterns in unlabeled data. Our approach can be integrated into established SSOD methods and is empirically validated using two baseline methods, with and without cascade regression, to generate more reliable pseudo-labels. Extensive experiments demonstrate the superior performance of our approach over state-of-the-art SSOD methods. Specifically, the proposed approach outperforms the baseline Unbiased-Teacher-v2 (& Unbiased-Teacher-v1) method by an average mAP margin of 2.23, 2.1, and 3.36 (& 2.07, 1.9, and 3.27) on COCO-standard, COCO-additional, and Pascal VOC datasets, respectively.