# Repetitive Reprediction Deep Decipher for Semi-Supervised Learning

arXiv: https://arxiv.org/abs/1908.04345 · HTML: https://mlsanity.com/paper/1908.04345/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2019.

## Built on by (most cited first, 11 of 11)

- [In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning](https://mlsanity.com/paper/2101.06329/) (2021)
- [Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervision](https://mlsanity.com/paper/2201.07929/) (2022)
- [SSMD: Semi-Supervised Medical Image Detection with Adaptive Consistency and Heterogeneous Perturbation](https://mlsanity.com/paper/2106.01544/) (2021)
- [All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation](https://mlsanity.com/paper/2305.15832/) (2023)
- [An Empirical Study and Analysis on Open-Set Semi-Supervised Learning](https://mlsanity.com/paper/2101.08237/) (2021)
- [A Simple and Efficient Pipeline to Build an End-to-End Spatial-Temporal Action Detector](https://mlsanity.com/paper/2206.03064/) (2022)
- [An Effective Approach for Multi-label Classification with Missing Labels](https://mlsanity.com/paper/2210.13651/) (2022)
- [PENCIL: Deep Learning with Noisy Labels](https://mlsanity.com/paper/2202.08436/) (2022)
- [R2-D2: Repetitive Reprediction Deep Decipher for Semi-Supervised Deep Learning](https://mlsanity.com/paper/2202.08955/) (2022)
- [Towards Modality-agnostic Label-efficient Segmentation with Entropy-Regularized Distribution Alignment](https://mlsanity.com/paper/2408.16520/) (2024)
- [CurvNet: Latent Contour Representation and Iterative Data Engine for Curvature Angle Estimation](https://mlsanity.com/paper/2411.12604/) (2024)

## Abstract (arXiv)

Most recent semi-supervised deep learning (deep SSL) methods used a similar paradigm: use network predictions to update pseudo-labels and use pseudo-labels to update network parameters iteratively. However, they lack theoretical support and cannot explain why predictions are good candidates for pseudo-labels. In this paper, we propose a principled end-to-end framework named deep decipher (D2) for SSL. Within the D2 framework, we prove that pseudo-labels are related to network predictions by an exponential link function, which gives a theoretical support for using predictions as pseudo-labels. Furthermore, we demonstrate that updating pseudo-labels by network predictions will make them uncertain. To mitigate this problem, we propose a training strategy called repetitive reprediction (R2). Finally, the proposed R2-D2 method is tested on the large-scale ImageNet dataset and outperforms state-of-the-art methods by 5 percentage points.