# Stratified Sampling for Model-Assisted Estimation with Surrogate Outcomes

arXiv: https://arxiv.org/abs/2602.12992 · HTML: https://mlsanity.com/paper/2602.12992/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2026.

## Built on by (most cited first, 3 of 3)

- [PPI is the Difference Estimator: Recognizing the Survey Sampling Roots of Prediction-Powered Inference](https://mlsanity.com/paper/2603.19160/) (2026)
- [Augmenting Human Evaluation with LLM Judges: How Many Human Reviews Do You Need?](https://mlsanity.com/paper/2605.16354/) (2026)
- [The Surrogate Is Not the Reward: Post-Surrogate Primary-Outcome Acquisition in Contextual Bandits](https://mlsanity.com/paper/2610.06610/) (2026)

## Abstract (arXiv)

In many randomized trials, outcomes such as essays or open-ended responses must be manually scored as a preliminary step to impact analysis, a process that is costly and limiting. Model-assisted estimation offers a way to combine surrogate outcomes generated by machine learning or large language models with a human-coded subset, yet typical implementations use simple random sampling and therefore overlook systematic variation in surrogate prediction error. We extend this framework by incorporating stratified sampling to more efficiently allocate human coding effort. We derive the exact variance of the stratified model-assisted estimator, characterize conditions under which stratification improves precision, and identify a Neyman-type optimal allocation rule that oversamples strata with larger residual variance. We evaluate our methods through a comprehensive simulation study to assess finite-sample performance. Overall, we find stratification consistently improves efficiency when surrogate prediction errors exhibit structured bias or heteroskedasticity. We also present two empirical applications, one using data from an education RCT and one using a large observational corpus, to illustrate how these methods can be implemented in practice using ChatGPT-generated surrogate outcomes. Overall, this framework provides a practical design-based approach for leveraging surrogate outcomes and strategically allocating human coding effort to obtain unbiased estimates with greater efficiency. While motivated by text-as-data applications, the methodology applies broadly to any setting where outcome measurement is costly or prohibitive, and can be applied to comparisons across groups or estimating the mean of a single group.