# Doubly Robust Self-Training

arXiv: https://arxiv.org/abs/2306.00265 · HTML: https://mlsanity.com/paper/2306.00265/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 55 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring](https://mlsanity.com/paper/1911.09785/) (2019)
- [Unbiased Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2102.09480/) (2021)
- [A Simple Semi-Supervised Learning Framework for Object Detection](https://mlsanity.com/paper/2005.04757/) (2020)
- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [DaViT: Dual Attention Vision Transformers](https://mlsanity.com/paper/2204.03645/) (2022)
- [Orthogonal Statistical Learning](https://mlsanity.com/paper/1901.09036/) (2019)
- [Offboard 3D Object Detection from Point Cloud Sequences](https://mlsanity.com/paper/2103.05073/) (2021)
- [Humble Teachers Teach Better Students for Semi-Supervised Object Detection](https://mlsanity.com/paper/2106.10456/) (2021)
- [3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/2012.04355/) (2020)
- [Dense Teacher: Dense Pseudo-Labels for Semi-supervised Object Detection](https://mlsanity.com/paper/2207.02541/) (2022)
- [Semi-supervised Inference: General Theory and Estimation of Means](https://mlsanity.com/paper/1606.07268/) (2016)
- [De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers](https://mlsanity.com/paper/1802.08667/) (2018)
- [Hierarchical Supervision and Shuffle Data Augmentation for 3D Semi-Supervised Object Detection](https://mlsanity.com/paper/2304.01464/) (2023)
- [DetMatch: Two Teachers are Better Than One for Joint 2D and 3D Semi-Supervised Object Detection](https://mlsanity.com/paper/2203.09510/) (2022)
- [Improving the Intra-class Long-tail in 3D Detection via Rare Example Mining](https://mlsanity.com/paper/2210.08375/) (2022)
- [DDS3D: Dense Pseudo-Labels with Dynamic Threshold for Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/2303.05079/) (2023)

## Built on by (most cited first, 18 of 18)

- [Semi-Supervised Risk Control via Prediction-Powered Inference](https://mlsanity.com/paper/2412.11174/) (2024)
- [Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning](https://mlsanity.com/paper/2504.03784/) (2025)
- [How to Bridge the Sim-to-Real Gap in Digital Twin-Aided Telecommunication Networks](https://mlsanity.com/paper/2507.07067/) (2025)
- [Semi-Supervised U-statistics](https://mlsanity.com/paper/2402.18921/) (2024)
- [Prediction-Powered Semi-Supervised Learning with Online Power Tuning](https://mlsanity.com/paper/2510.22586/) (2025)
- [Automated Efficient Estimation using Monte Carlo Efficient Influence Functions](https://mlsanity.com/paper/2403.00158/) (2024)
- [Automatic AI Model Selection for Wireless Systems: Online Learning via Digital Twinning](https://mlsanity.com/paper/2406.15819/) (2024)
- [Do We Really Even Need Data? A Modern Look at Drawing Inference with Predicted Data](https://mlsanity.com/paper/2512.05456/) (2025)
- [Context-Aware Doubly-Robust Semi-Supervised Learning](https://mlsanity.com/paper/2502.15577/) (2025)
- [Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective](https://mlsanity.com/paper/2512.05267/) (2025)
- [Enhancing Semi-supervised Learning with Zero-shot Pseudolabels](https://mlsanity.com/paper/2502.12584/) (2025)
- [SADA: Safe and Adaptive Aggregation of Multiple Black-Box Predictions in Semi-Supervised Learning](https://mlsanity.com/paper/2509.21707/) (2025)
- [GeoTeacher: Geometry-Guided Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/2512.23147/) (2025)
- [DeDPO: Debiased Direct Preference Optimization for Diffusion Models](https://mlsanity.com/paper/2602.06195/) (2026)
- [TrajSSL: Trajectory-Enhanced Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/2409.10901/) (2024)
- [Prediction-Augmented Trees for Reliable Statistical Inference](https://mlsanity.com/paper/2510.16937/) (2025)
- [Learning U-Statistics with Active Inference](https://mlsanity.com/paper/2605.11638/) (2026)
- [ABC-Align: Prediction-Powered Alignment with Adaptive Bias Control](https://mlsanity.com/paper/2609.34374/) (2026)

## Abstract (arXiv)

Self-training is an important technique for solving semi-supervised learning problems. It leverages unlabeled data by generating pseudo-labels and combining them with a limited labeled dataset for training. The effectiveness of self-training heavily relies on the accuracy of these pseudo-labels. In this paper, we introduce doubly robust self-training, a novel semi-supervised algorithm that provably balances between two extremes. When the pseudo-labels are entirely incorrect, our method reduces to a training process solely using labeled data. Conversely, when the pseudo-labels are completely accurate, our method transforms into a training process utilizing all pseudo-labeled data and labeled data, thus increasing the effective sample size. Through empirical evaluations on both the ImageNet dataset for image classification and the nuScenes autonomous driving dataset for 3D object detection, we demonstrate the superiority of the doubly robust loss over the standard self-training baseline.