mlsanityData to 7 Oct 2026

Benchmarking Framework for Performance-Evaluation of Causal Inference Analysis

arXiv 1802.05046 ↗First posted 14 Feb 2018stat.ME cs.LG stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

Causal inference analysis is the estimation of the effects of actions on outcomes. In the context of healthcare data this means estimating the outcome of counter-factual treatments (i.e. including treatments that were not observed) on a patient's outcome. Compared to classic machine learning methods, evaluation and validation of causal inference analysis is more challenging because ground truth data of counter-factual outcome can never be obtained in any real-world scenario. Here, we present a comprehensive framework for benchmarking algorithms that estimate causal effect. The framework includes unlabeled data for prediction, labeled data for validation, and code for automatic evaluation of algorithm predictions using both established and novel metrics. The data is based on real-world covariates, and the treatment assignments and outcomes are based on simulations, which provides the basis for validation. In this framework we address two questions: one of scaling, and the other of data-censoring. The framework is available as open source code at https://github.com/IBM-HRL-MLHLS/IBM-Causal-Inference-Benchmarking-Framework

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 6; 2023: 7; 2024: 5; 2025: 8; 2026: 1.

33 papers.

Sort:

By area:

  1. 2022Diffusion Causal Models for Counterfactual Estimationcited by 87
  2. 2025CausalPFN: Amortized Causal Effect Estimation via In-Context Learningcited by 43
  3. 2022CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Datacited by 29
  4. 2021Estimating Average Treatment Effects via Orthogonal Regularizationcited by 25
  5. 2022Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimationcited by 25
  6. 2025CausalRivers -- Scaling up benchmarking of causal discovery for real-world time-seriescited by 24
  7. 2022Evaluation Methods and Measures for Causal Learning Algorithmscited by 23
  8. 2022Normalizing Flows for Interventional Density Estimationcited by 21
  9. 2024Causal Evaluation of Language Modelscited by 18
  10. 2023Zero-shot causal learningcited by 17
  11. 2023Towards Causal Foundation Model: on Duality between Causal Inference and Attentioncited by 17
  12. 2022A Survey of Deep Causal Models and Their Industrial Applicationscited by 10
21 more
  1. 2021Generating Synthetic Text Data to Evaluate Causal Inference Methodscited by 9
  2. 2023RCT Rejection Sampling for Causal Estimation Evaluationcited by 7
  3. 2024Proximal Causal Inference With Text Datacited by 7
  4. 2025Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoptioncited by 6
  5. 2021Bayesian Model Averaging for Causality Estimation and its Approximation based on Gaussian Scale Mixture Distributionscited by 4
  6. 2021Causal Effect Estimation using Variational Information Bottleneckcited by 4
  7. 2023Uncertainty Quantification in Heterogeneous Treatment Effect Estimation with Gaussian-Process-Based Partially Linear Modelcited by 4
  8. 2025CausalMan: A physics-based simulator for large-scale causalitycited by 4
  9. 2023Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimationcited by 3
  10. 2024Progressive Generalization Risk Reduction for Data-Efficient Causal Effect Estimationcited by 3
  11. 2025Enhancing Treatment Effect Estimation via Active Learning: A Counterfactual Covering Perspectivecited by 3
  12. 2023How to select predictive models for causal inference?cited by 2
  13. 2024Differentiable Pareto-Smoothed Weighting for High-Dimensional Heterogeneous Treatment Effect Estimationcited by 2
  14. 2021Identifying Causal Influences on Publication Trends and Behavior: A Case Study of the Computational Linguistics Communitycited by 1
  15. 2023Integrating Nearest Neighbors with Neural Network Models for Treatment Effect Estimationcited by 1
  16. 2025Flow-based Generative Modeling of Potential Outcomes and Counterfactualscited by 1
  17. 2021ADCB: An Alzheimer's disease benchmark for evaluating observational estimators of causal effectscited by 0
  18. 2024C-XGBoost: A tree boosting model for causal effect estimationcited by 0
  19. 2025A Computational Theory for Efficient Mini Agent Evaluation with Causal Guaranteescited by 0
  20. 2025Improving Generative Methods for Causal Evaluation via Simulation-Based Inferencecited by 0
  21. 2026Controllable Generative Sandbox for Causal Inferencecited by 0