# An evaluation framework for comparing causal inference models

arXiv: https://arxiv.org/abs/2209.00115 · HTML: https://mlsanity.com/paper/2209.00115/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 30 references that are in the map.
- First posted on arXiv 31 Aug 2022.

## Builds on (most cited first, 4 of 4)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)

## Built on by (most cited first, 4 of 4)

- [Integrating Nearest Neighbors with Neural Network Models for Treatment Effect Estimation](https://mlsanity.com/paper/2305.06789/) (2023)
- [MOSIC: Model-Agnostic Optimal Subgroup Identification with Multi-Constraint for Improved Reliability](https://mlsanity.com/paper/2504.20908/) (2025)
- [C-XGBoost: A tree boosting model for causal effect estimation](https://mlsanity.com/paper/2404.00751/) (2024)
- [Testing Generalizability in Causal Inference](https://mlsanity.com/paper/2411.03021/) (2024)

## Abstract (arXiv)

Estimation of causal effects is the core objective of many scientific disciplines. However, it remains a challenging task, especially when the effects are estimated from observational data. Recently, several promising machine learning models have been proposed for causal effect estimation. The evaluation of these models has been based on the mean values of the error of the Average Treatment Effect (ATE) as well as of the Precision in Estimation of Heterogeneous Effect (PEHE). In this paper, we propose to complement the evaluation of causal inference models using concrete statistical evidence, including the performance profiles of Dolan and Mor'e, as well as non-parametric and post-hoc statistical tests. The main motivation behind this approach is the elimination of the influence of a small number of instances or simulation on the benchmarking process, which in some cases dominate the results. We use the proposed evaluation methodology to compare several state-of-the-art causal effect estimation models.