# Predictions as Surrogates: Revisiting Surrogate Outcomes in the Age of AI

arXiv: https://arxiv.org/abs/2501.09731 · HTML: https://mlsanity.com/paper/2501.09731/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 73 references that are in the map.
- First posted on arXiv 16 Jan 2025.

## Builds on (most cited first, 28 of 28)

- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [Prediction-Powered Inference](https://mlsanity.com/paper/2301.09633/) (2023)
- [Semiparametric doubly robust targeted double machine learning: a review](https://mlsanity.com/paper/2203.06469/) (2022)
- [TorchXRayVision: A library of chest X-ray datasets and models](https://mlsanity.com/paper/2111.00595/) (2021)
- [PPI++: Efficient Prediction-Powered Inference](https://mlsanity.com/paper/2311.01453/) (2023)
- [A Computational Approach to Politeness with Application to Social Factors](https://mlsanity.com/paper/1306.6078/) (2013)
- [Active Statistical Inference](https://mlsanity.com/paper/2403.03208/) (2024)
- [Can Unconfident LLM Annotations Be Used for Confident Conclusions?](https://mlsanity.com/paper/2408.15204/) (2024)
- [Using Imperfect Surrogates for Downstream Inference: Design-based Supervised Learning for Social Science Applications of Large Language Models](https://mlsanity.com/paper/2306.04746/) (2023)
- [Semi-supervised Inference: General Theory and Estimation of Means](https://mlsanity.com/paper/1606.07268/) (2016)
- [On the role of surrogates in the efficient estimation of treatment effects with limited outcome data](https://mlsanity.com/paper/2003.12408/) (2020)
- [Efficient and Adaptive Linear Regression in Semi-Supervised Settings](https://mlsanity.com/paper/1701.04889/) (2017)
- [Combining multiple observational data sources to estimate causal effects](https://mlsanity.com/paper/1801.00802/) (2018)
- [Assumption-Lean and Data-Adaptive Post-Prediction Inference](https://mlsanity.com/paper/2311.14220/) (2023)
- [Prediction De-Correlated Inference: A safe approach for post-prediction inference](https://mlsanity.com/paper/2312.06478/) (2023)
- [CAREER: A Foundation Model for Labor Sequence Data](https://mlsanity.com/paper/2202.08370/) (2022)
- [A Note on the Prediction-Powered Bootstrap](https://mlsanity.com/paper/2405.18379/) (2024)
- [Task-Agnostic Machine-Learning-Assisted Inference](https://mlsanity.com/paper/2405.20039/) (2024)
- [Inferring the Long-Term Causal Effects of Long-Term Treatments from Short-Term Experiments](https://mlsanity.com/paper/2311.08527/) (2023)
- [Bayesian Prediction-Powered Inference](https://mlsanity.com/paper/2405.06034/) (2024)
- [Evaluating the Surrogate Index as a Decision-Making Tool Using 200 A/B Tests at Netflix](https://mlsanity.com/paper/2311.11922/) (2023)
- [LABOR-LLM: Language-Based Occupational Representations with Large Language Models](https://mlsanity.com/paper/2406.17972/) (2024)

## Built on by (most cited first, 30 of 34)

- [Multi-Armed Bandits With Machine Learning-Generated Surrogate Rewards](https://mlsanity.com/paper/2506.16658/) (2025)
- [Efficient Randomized Experiments Using Foundation Models](https://mlsanity.com/paper/2502.04262/) (2025)
- [Demystifying Prediction Powered Inference](https://mlsanity.com/paper/2601.20819/) (2026)
- [AI-Powered Bayesian Inference](https://mlsanity.com/paper/2502.19231/) (2025)
- [Causal Inference on Outcomes Learned from Text](https://mlsanity.com/paper/2503.00725/) (2025)
- [Doubly-Robust LLM-as-a-Judge: Externally Valid Estimation with Imperfect Personas](https://mlsanity.com/paper/2509.22957/) (2025)
- [Valid Inference with Imperfect Synthetic Data](https://mlsanity.com/paper/2508.06635/) (2025)
- [A Unified Framework for Inference with General Missingness Patterns and Machine Learning Imputation](https://mlsanity.com/paper/2508.15162/) (2025)
- [Efficient Inference for Noisy LLM-as-a-Judge Evaluation](https://mlsanity.com/paper/2601.05420/) (2026)
- [General Synthetic-Powered Inference](https://mlsanity.com/paper/2509.20345/) (2025)
- [Robust Sampling for Active Statistical Inference](https://mlsanity.com/paper/2511.08991/) (2025)
- [Imputation-Powered Inference](https://mlsanity.com/paper/2509.13778/) (2025)
- [Prediction-Powered Semi-Supervised Learning with Online Power Tuning](https://mlsanity.com/paper/2510.22586/) (2025)
- [Revisiting Active Sequential Prediction-Powered Mean Estimation](https://mlsanity.com/paper/2604.18569/) (2026)
- [Calibeating Prediction-Powered Inference](https://mlsanity.com/paper/2604.21260/) (2026)
- [Do We Really Even Need Data? A Modern Look at Drawing Inference with Predicted Data](https://mlsanity.com/paper/2512.05456/) (2025)
- [Harnessing Synthetic Data from Generative AI for Statistical Inference](https://mlsanity.com/paper/2603.05396/) (2026)
- [Prediction-Powered Inference Across Many Tasks for AI Evaluation & Social Science Research](https://mlsanity.com/paper/2605.29249/) (2026)
- [Extending Prediction-Powered Inference through Conformal Prediction](https://mlsanity.com/paper/2510.16166/) (2025)
- [SADA: Safe and Adaptive Aggregation of Multiple Black-Box Predictions in Semi-Supervised Learning](https://mlsanity.com/paper/2509.21707/) (2025)
- [Scalable Vision-Guided Crop Yield Estimation](https://mlsanity.com/paper/2511.12999/) (2025)
- [Allocating Human Oversight in AI-Enabled Analytics](https://mlsanity.com/paper/2604.12497/) (2026)
- [Industrializing Prediction-Powered Inference: The GLIDE Library for Reliable GenAI and Agentic Systems Evaluation](https://mlsanity.com/paper/2605.31278/) (2026)
- [Optimized Labeling Resource Allocation for Prediction-Assisted Inference via OPAL](https://mlsanity.com/paper/2606.03211/) (2026)
- [Valid Inference with Synthetic Data via Task Exchangeability](https://mlsanity.com/paper/2606.13629/) (2026)
- [Which Metrics Save the Most Human Annotation? Prediction-Powered Evaluation and Meta-Evaluation](https://mlsanity.com/paper/2608.26638/) (2026)
- [Calibrated Principal Component Regression](https://mlsanity.com/paper/2510.19020/) (2025)
- [Personalizing black-box models for nonparametric regression with minimax optimality](https://mlsanity.com/paper/2601.01432/) (2026)
- [AI-Generated Measurements for Identification and Inference with Missing Data: A Weak Shadow Variable Approach](https://mlsanity.com/paper/2602.16061/) (2026)
- [Rectification Difficulty and Optimal Sample Allocation in LLM-Augmented Surveys](https://mlsanity.com/paper/2604.17267/) (2026)

## Abstract (arXiv)

We establish a formal connection between the decades-old surrogate outcome model in biostatistics and economics and the emerging field of prediction-powered inference (PPI). The connection treats predictions from pre-trained models, prevalent in the age of AI, as cost-effective surrogates for expensive outcomes. Building on the surrogate outcomes literature, we develop recalibrated prediction-powered inference, a more efficient approach to statistical inference than existing PPI proposals. Our method departs from the existing proposals by using flexible machine learning techniques to learn the optimal ``imputed loss'' through a step we call recalibration. Importantly, the method always improves upon the estimator that relies solely on the data with available true outcomes, even when the optimal imputed loss is estimated imperfectly, and it achieves the smallest asymptotic variance among PPI estimators if the estimate is consistent. Computationally, our optimization objective is convex whenever the loss function that defines the target parameter is convex. We further analyze the benefits of recalibration, both theoretically and numerically, in several common scenarios where machine learning predictions systematically deviate from the outcome of interest. We demonstrate significant gains in effective sample size over existing PPI proposals via three applications leveraging state-of-the-art machine learning/AI models.