# High Precision Causal Model Evaluation with Conditional Randomization

arXiv: https://arxiv.org/abs/2311.01902 · HTML: https://mlsanity.com/paper/2311.01902/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 44 references that are in the map.
- First posted on arXiv 3 Nov 2023.

## Builds on (most cited first, 11 of 11)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Recursive Partitioning for Heterogeneous Causal Effects](https://mlsanity.com/paper/1504.01132/) (2015)
- [Deep End-to-end Causal Inference](https://mlsanity.com/paper/2202.02195/) (2022)
- [Orthogonal Random Forest for Causal Inference](https://mlsanity.com/paper/1806.03467/) (2018)
- [Approximate Residual Balancing: De-Biased Inference of Average Treatment Effects in High Dimensions](https://mlsanity.com/paper/1604.07125/) (2016)
- [High-dimensional regression adjustments in randomized experiments](https://mlsanity.com/paper/1607.06801/) (2016)
- [Estimation and Inference for Moments of Ratios with Robustness against Large Trimming Bias](https://mlsanity.com/paper/1709.00981/) (2017)
- [Variance Reduction for Causal Inference](https://mlsanity.com/paper/2109.05150/) (2021)

## Built on by (most cited first, 1 of 1)

- [The Essential Role of Causality in Foundation World Models for Embodied AI](https://mlsanity.com/paper/2402.06665/) (2024)

## Abstract (arXiv)

The gold standard for causal model evaluation involves comparing model predictions with true effects estimated from randomized controlled trials (RCT). However, RCTs are not always feasible or ethical to perform. In contrast, conditionally randomized experiments based on inverse probability weighting (IPW) offer a more realistic approach but may suffer from high estimation variance. To tackle this challenge and enhance causal model evaluation in real-world conditional randomization settings, we introduce a novel low-variance estimator for causal error, dubbed as the pairs estimator. By applying the same IPW estimator to both the model and true experimental effects, our estimator effectively cancels out the variance due to IPW and achieves a smaller asymptotic variance. Empirical studies demonstrate the improved of our estimator, highlighting its potential on achieving near-RCT performance. Our method offers a simple yet powerful solution to evaluate causal inference models in conditional randomization settings without complicated modification of the IPW estimator itself, paving the way for more robust and reliable model assessments.