# Semi-supervised Inference: General Theory and Estimation of Means

arXiv: https://arxiv.org/abs/1606.07268 · HTML: https://mlsanity.com/paper/1606.07268/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2016.

## Built on by (most cited first, 30 of 38)

- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [PPI++: Efficient Prediction-Powered Inference](https://mlsanity.com/paper/2311.01453/) (2023)
- [Cross-Prediction-Powered Inference](https://mlsanity.com/paper/2309.16598/) (2023)
- [Active Statistical Inference](https://mlsanity.com/paper/2403.03208/) (2024)
- [Predictions as Surrogates: Revisiting Surrogate Outcomes in the Age of AI](https://mlsanity.com/paper/2501.09731/) (2025)
- [Assumption-Lean and Data-Adaptive Post-Prediction Inference](https://mlsanity.com/paper/2311.14220/) (2023)
- [Doubly Robust Self-Training](https://mlsanity.com/paper/2306.00265/) (2023)
- [Prediction De-Correlated Inference: A safe approach for post-prediction inference](https://mlsanity.com/paper/2312.06478/) (2023)
- [Multi-Armed Bandits With Machine Learning-Generated Surrogate Rewards](https://mlsanity.com/paper/2506.16658/) (2025)
- [Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning](https://mlsanity.com/paper/2504.03784/) (2025)
- [Semi-Supervised Quantile Estimation: Robust and Efficient Inference in High Dimensional Settings](https://mlsanity.com/paper/2201.10208/) (2022)
- [Task-Agnostic Machine-Learning-Assisted Inference](https://mlsanity.com/paper/2405.20039/) (2024)
- [Local Prediction-Powered Inference](https://mlsanity.com/paper/2409.18321/) (2024)
- [Regression for the Mean: Auto-Evaluation and Inference with Few Labels through Post-hoc Regression](https://mlsanity.com/paper/2411.12665/) (2024)
- [Another look at statistical inference with machine learning-imputed data](https://mlsanity.com/paper/2411.19908/) (2024)
- [FAB-PPI: Frequentist, Assisted by Bayes, Prediction-Powered Inference](https://mlsanity.com/paper/2502.02363/) (2025)
- [Anytime-valid, Bayes-assisted, Prediction-Powered Inference](https://mlsanity.com/paper/2505.18000/) (2025)
- [Surrogate Assisted Semi-supervised Inference for High Dimensional Risk Prediction](https://mlsanity.com/paper/2105.01264/) (2021)
- [Efficient and Multiply Robust Risk Estimation under General Forms of Dataset Shift](https://mlsanity.com/paper/2306.16406/) (2023)
- [Prediction-Powered Semi-Supervised Learning with Online Power Tuning](https://mlsanity.com/paper/2510.22586/) (2025)
- [Double Robust Semi-Supervised Inference for the Mean: Selection Bias under MAR Labeling with Decaying Overlap](https://mlsanity.com/paper/2104.06667/) (2021)
- [Sharp-SSL: Selective high-dimensional axis-aligned random projections for semi-supervised learning](https://mlsanity.com/paper/2304.09154/) (2023)
- [Mixed Semi-Supervised Generalized-Linear-Regression with Applications to Deep-Learning and Interpolators](https://mlsanity.com/paper/2302.09526/) (2023)
- [Distributed Semi-Supervised Sparse Statistical Inference](https://mlsanity.com/paper/2306.10395/) (2023)
- [On semi-supervised estimation using exponential tilt mixture models](https://mlsanity.com/paper/2311.08504/) (2023)
- [Efficient semi-supervised inference for logistic regression under case-control studies](https://mlsanity.com/paper/2402.15365/) (2024)
- [Bayesian Semi-supervised Inference via a Debiased Modeling Approach](https://mlsanity.com/paper/2509.17385/) (2025)
- [SADA: Safe and Adaptive Aggregation of Multiple Black-Box Predictions in Semi-Supervised Learning](https://mlsanity.com/paper/2509.21707/) (2025)
- [MEC: Machine-Learning-Assisted Generalized Entropy Calibration for Semi-Supervised Mean Estimation](https://mlsanity.com/paper/2604.05446/) (2026)
- [High-dimensional Semi-supervised Classification via the Fermat Distance](https://mlsanity.com/paper/2604.23573/) (2026)

## Abstract (arXiv)

We propose a general semi-supervised inference framework focused on the estimation of the population mean. As usual in semi-supervised settings, there exists an unlabeled sample of covariate vectors and a labeled sample consisting of covariate vectors along with real-valued responses ("labels"). Otherwise, the formulation is "assumption-lean" in that no major conditions are imposed on the statistical or functional form of the data. We consider both the ideal semi-supervised setting where infinitely many unlabeled samples are available, as well as the ordinary semi-supervised setting in which only a finite number of unlabeled samples is available. Estimators are proposed along with corresponding confidence intervals for the population mean. Theoretical analysis on both the asymptotic distribution and ℓ2-risk for the proposed procedures are given. Surprisingly, the proposed estimators, based on a simple form of the least squares method, outperform the ordinary sample mean. The simple, transparent form of the estimator lends confidence to the perception that its asymptotic improvement over the ordinary sample mean also nearly holds even for moderate size samples. The method is further extended to a nonparametric setting, in which the oracle rate can be achieved asymptotically. The proposed estimators are further illustrated by simulation studies and a real data example involving estimation of the homeless population.