# Using Experiments to Correct for Selection in Observational Studies

arXiv: https://arxiv.org/abs/2006.09676 · HTML: https://mlsanity.com/paper/2006.09676/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2020.

## Built on by (most cited first, 28 of 28)

- [Off-Policy Evaluation for Large Action Spaces via Embeddings](https://mlsanity.com/paper/2202.06317/) (2022)
- [Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling](https://mlsanity.com/paper/2305.08062/) (2023)
- [Combining Observational and Randomized Data for Estimating Heterogeneous Treatment Effects](https://mlsanity.com/paper/2202.12891/) (2022)
- [Long-term Causal Inference Under Persistent Confounding via Data Combination](https://mlsanity.com/paper/2202.07234/) (2022)
- [Causal Decision Making and Causal Effect Estimation Are Not the Same... and Why It Matters](https://mlsanity.com/paper/2104.04103/) (2021)
- [Federated Causal Inference in Heterogeneous Observational Data](https://mlsanity.com/paper/2107.11732/) (2021)
- [Detecting hidden confounding in observational data using multiple environments](https://mlsanity.com/paper/2205.13935/) (2022)
- [POTEC: Off-Policy Learning for Large Action Spaces via Two-Stage Policy Decomposition](https://mlsanity.com/paper/2402.06151/) (2024)
- [Long-term Causal Effects Estimation via Latent Surrogates Representation Learning](https://mlsanity.com/paper/2208.04589/) (2022)
- [Meta-Learners for Partially-Identified Treatment Effects Across Multiple Environments](https://mlsanity.com/paper/2406.02464/) (2024)
- [Long-term Off-Policy Evaluation and Learning](https://mlsanity.com/paper/2404.15691/) (2024)
- [Policy Learning for Balancing Short-Term and Long-Term Rewards](https://mlsanity.com/paper/2405.03329/) (2024)
- [A Reinforcement Learning Approach to Estimating Long-term Treatment Effects](https://mlsanity.com/paper/2210.07536/) (2022)
- [Kernel methods for long term dose response curves](https://mlsanity.com/paper/2201.05139/) (2022)
- [Estimating Long-term Heterogeneous Dose-response Curve: Generalization Bound Leveraging Optimal Transport Weights](https://mlsanity.com/paper/2406.19195/) (2024)
- [Estimating Heterogeneous Treatment Effects by Combining Weak Instruments and Observational Data](https://mlsanity.com/paper/2406.06452/) (2024)
- [Fast Proxy Experiment Design for Causal Effect Identification](https://mlsanity.com/paper/2407.05330/) (2024)
- [On the Role of Surrogates in Conformal Inference of Individual Causal Effects](https://mlsanity.com/paper/2412.12365/) (2024)
- [Nonparametric Heterogeneous Long-term Causal Effect Estimation via Data Combination](https://mlsanity.com/paper/2502.18960/) (2025)
- [Calibrated Optimal Decision Making with Multiple Data Sources and Limited Outcome](https://mlsanity.com/paper/2104.10554/) (2021)
- [Causal inference with Machine Learning-Based Covariate Representation](https://mlsanity.com/paper/2311.01709/) (2023)
- [Identification and Estimation for Nonignorable Missing Data: A Data Fusion Approach](https://mlsanity.com/paper/2311.09015/) (2023)
- [Combining Incomplete Observational and Randomized Data for Heterogeneous Treatment Effects](https://mlsanity.com/paper/2410.21343/) (2024)
- [Statistical Foundations of LLM-based A/B Testing: A Surrogacy Framework for Human Causal Inference](https://mlsanity.com/paper/2606.17165/) (2026)
- [Combining Interventional and Observational Data Using Causal Reductions](https://mlsanity.com/paper/2103.04786/) (2021)
- [Pareto-Optimal Estimation and Policy Learning on Short-term and Long-term Treatment Effects](https://mlsanity.com/paper/2403.02624/) (2024)
- [Online Data Collection for Efficient Semiparametric Inference](https://mlsanity.com/paper/2411.03195/) (2024)
- [A General Framework for Budgeted Threshold Incentives on Request](https://mlsanity.com/paper/2609.29724/) (2026)

## Abstract (arXiv)

Researchers increasingly have access to two types of data: (i) large observational datasets where treatment (e.g., class size) is not randomized but several primary outcomes (e.g., graduation rates) and secondary outcomes (e.g., test scores) are observed and (ii) experimental data in which treatment is randomized but only secondary outcomes are observed. We develop a new method to estimate treatment effects on primary outcomes in such settings. We use the difference between the secondary outcome and its predicted value based on the experimental treatment effect to measure selection bias in the observational data. Controlling for this estimate of selection bias yields an unbiased estimate of the treatment effect on the primary outcome under a new assumption that we term latent unconfoundedness, which requires that the same confounders affect the primary and secondary outcomes. Latent unconfoundedness weakens the assumptions underlying commonly used surrogate estimators. We apply our estimator to identify the effect of third grade class size on students outcomes. Estimated impacts on test scores using OLS regressions in observational school district data have the opposite sign of estimates from the Tennessee STAR experiment. In contrast, selection-corrected estimates in the observational data replicate the experimental estimates. Our estimator reveals that reducing class sizes by 25% increases high school graduation rates by 0.7 percentage points. Controlling for observables does not change the OLS estimates, demonstrating that experimental selection correction can remove biases that cannot be addressed with standard controls.