# Efficient and Adaptive Linear Regression in Semi-Supervised Settings

arXiv: https://arxiv.org/abs/1701.04889 · HTML: https://mlsanity.com/paper/1701.04889/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2017.

## Built on by (most cited first, 29 of 29)

- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [Using Imperfect Surrogates for Downstream Inference: Design-based Supervised Learning for Social Science Applications of Large Language Models](https://mlsanity.com/paper/2306.04746/) (2023)
- [Predictions as Surrogates: Revisiting Surrogate Outcomes in the Age of AI](https://mlsanity.com/paper/2501.09731/) (2025)
- [Assumption-Lean and Data-Adaptive Post-Prediction Inference](https://mlsanity.com/paper/2311.14220/) (2023)
- [Prediction De-Correlated Inference: A safe approach for post-prediction inference](https://mlsanity.com/paper/2312.06478/) (2023)
- [TabPFN: One Model to Rule Them All?](https://mlsanity.com/paper/2505.20003/) (2025)
- [Semi-Supervised Risk Control via Prediction-Powered Inference](https://mlsanity.com/paper/2412.11174/) (2024)
- [Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning](https://mlsanity.com/paper/2504.03784/) (2025)
- [Semi-Supervised Quantile Estimation: Robust and Efficient Inference in High Dimensional Settings](https://mlsanity.com/paper/2201.10208/) (2022)
- [Local Prediction-Powered Inference](https://mlsanity.com/paper/2409.18321/) (2024)
- [Surrogate Assisted Semi-supervised Inference for High Dimensional Risk Prediction](https://mlsanity.com/paper/2105.01264/) (2021)
- [Efficient and Multiply Robust Risk Estimation under General Forms of Dataset Shift](https://mlsanity.com/paper/2306.16406/) (2023)
- [Double Robust Semi-Supervised Inference for the Mean: Selection Bias under MAR Labeling with Decaying Overlap](https://mlsanity.com/paper/2104.06667/) (2021)
- [Adaptive Semi-Supervised Inference for Optimal Treatment Decisions with Electronic Medical Record Data](https://mlsanity.com/paper/2203.02318/) (2022)
- [Semi-supervised Batch Learning From Logged Data](https://mlsanity.com/paper/2209.07148/) (2022)
- [Sharp-SSL: Selective high-dimensional axis-aligned random projections for semi-supervised learning](https://mlsanity.com/paper/2304.09154/) (2023)
- [GEAR: On Optimal Decision Making with Auxiliary Data](https://mlsanity.com/paper/2104.10573/) (2021)
- [Mixed Semi-Supervised Generalized-Linear-Regression with Applications to Deep-Learning and Interpolators](https://mlsanity.com/paper/2302.09526/) (2023)
- [Distributed Semi-Supervised Sparse Statistical Inference](https://mlsanity.com/paper/2306.10395/) (2023)
- [Efficient semi-supervised inference for logistic regression under case-control studies](https://mlsanity.com/paper/2402.15365/) (2024)
- [Bayesian Semi-supervised Inference via a Debiased Modeling Approach](https://mlsanity.com/paper/2509.17385/) (2025)
- [SADA: Safe and Adaptive Aggregation of Multiple Black-Box Predictions in Semi-Supervised Learning](https://mlsanity.com/paper/2509.21707/) (2025)
- [Coupled Training with Privileged Information and Unlabeled Data](https://mlsanity.com/paper/2605.23268/) (2026)
- [Semi-supervised Fr'echet Regression](https://mlsanity.com/paper/2404.10444/) (2024)
- [Optimal Exact Recovery in Semi-Supervised Learning: A Study of Spectral Methods and Graph Convolutional Networks](https://mlsanity.com/paper/2412.13754/) (2024)
- [Semi-supervised learning for linear extremile regression](https://mlsanity.com/paper/2507.01314/) (2025)
- [Rescuing double robustness: safe estimation under complete misspecification](https://mlsanity.com/paper/2509.22446/) (2025)
- [Target-Aware Linear Regression Under Distribution Shift](https://mlsanity.com/paper/2606.22775/) (2026)
- [Semi-Supervised Conditional Diffusion via Label Augmentation](https://mlsanity.com/paper/2607.16685/) (2026)

## Abstract (arXiv)

We consider the linear regression problem under semi-supervised settings wherein the available data typically consists of: (i) a small or moderate sized 'labeled' data, and (ii) a much larger sized 'unlabeled' data. Such data arises naturally from settings where the outcome, unlike the covariates, is expensive to obtain, a frequent scenario in modern studies involving large databases like electronic medical records (EMR). Supervised estimators like the ordinary least squares (OLS) estimator utilize only the labeled data. It is often of interest to investigate if and when the unlabeled data can be exploited to improve estimation of the regression parameter in the adopted linear model. In this paper, we propose a class of 'Efficient and Adaptive Semi-Supervised Estimators' (EASE) to improve estimation efficiency. The EASE are two-step estimators adaptive to model mis-specification, leading to improved (optimal in some cases) efficiency under model mis-specification, and equal (optimal) efficiency under a linear model. This adaptive property, often unaddressed in the existing literature, is crucial for advocating 'safe' use of the unlabeled data. The construction of EASE primarily involves a flexible 'semi-non-parametric' imputation, including a smoothing step that works well even when the number of covariates is not small; and a follow up 'refitting' step along with a cross-validation (CV) strategy both of which have useful practical as well as theoretical implications towards addressing two important issues: under-smoothing and over-fitting. We establish asymptotic results including consistency, asymptotic normality and the adaptive properties of EASE. We also provide influence function expansions and a 'double' CV strategy for inference. The results are further validated through extensive simulations, followed by application to an EMR study on auto-immunity.