# Combining observational and experimental data to find heterogeneous treatment effects

arXiv: https://arxiv.org/abs/1611.02385 · HTML: https://mlsanity.com/paper/1611.02385/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2016.

## Built on by (most cited first, 8 of 8)

- [Causal Decision Making and Causal Effect Estimation Are Not the Same... and Why It Matters](https://mlsanity.com/paper/2104.04103/) (2021)
- [Causal Effect Variational Autoencoder with Uniform Treatment](https://mlsanity.com/paper/2111.08656/) (2021)
- [Efficient Heterogeneous Treatment Effect Estimation With Multiple Experiments and Multiple Outcomes](https://mlsanity.com/paper/2206.04907/) (2022)
- [Combining Experimental and Historical Data for Policy Evaluation](https://mlsanity.com/paper/2406.00317/) (2024)
- [Causal Inference with Latent Variables: Recent Advances and Future Prospectives](https://mlsanity.com/paper/2406.13966/) (2024)
- [A Fast Bootstrap Algorithm for Causal Inference with Large Data](https://mlsanity.com/paper/2302.02859/) (2023)
- [Decoding ML Decision: An Agentic Reasoning Framework for Large-Scale Ranking System](https://mlsanity.com/paper/2602.18640/) (2026)
- [Causal Post-Processing of Predictive Models](https://mlsanity.com/paper/2406.09567/) (2024)

## Abstract (arXiv)

Every design choice will have different effects on different units. However traditional A/B tests are often underpowered to identify these heterogeneous effects. This is especially true when the set of unit-level attributes is high-dimensional and our priors are weak about which particular covariates are important. However, there are often observational data sets available that are orders of magnitude larger. We propose a method to combine these two data sources to estimate heterogeneous treatment effects. First, we use observational time series data to estimate a mapping from covariates to unit-level effects. These estimates are likely biased but under some conditions the bias preserves unit-level relative rank orderings. If these conditions hold, we only need sufficient experimental data to identify a monotonic, one-dimensional transformation from observationally predicted treatment effects to real treatment effects. This reduces power demands greatly and makes the detection of heterogeneous effects much easier. As an application, we show how our method can be used to improve Facebook page recommendations.