mlsanityData to 7 Oct 2026

Bayesian Counterfactual Mean Embeddings and Off-Policy Evaluation

arXiv 2211.01518 ↗First posted 2 Nov 2022stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The counterfactual distribution models the effect of the treatment in the untreated group. While most of the work focuses on the expected values of the treatment effect, one may be interested in the whole counterfactual distribution or other quantities associated to it. Building on the framework of Bayesian conditional mean embeddings, we propose a Bayesian approach for modeling the counterfactual distribution, which leads to quantifying the epistemic uncertainty about the distribution. The framework naturally extends to the setting where one observes multiple treatment effects (e.g. an intermediate effect after an interim period, and an ultimate treatment effect which is of main interest) and allows for additionally modelling uncertainty about the relationship of these effects. For such goal, we present three novel Bayesian methods to estimate the expectation of the ultimate treatment effect, when only noisy samples of the dependence between intermediate and ultimate effects are provided. These methods differ on the source of uncertainty considered and allow for combining two sources of data. Moreover, we generalize these ideas to the off-policy evaluation framework, which can be seen as an extension of the counterfactual estimation problem. We empirically explore the calibration of the algorithms in two different experimental settings which require data fusion, and illustrate the value of considering the uncertainty stemming from the two sources of data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 49 references that are in the map. First posted on arXiv 2 Nov 2022.

Builds on

22 of 49 references.

References · 22

  1. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 360
  2. 2016Learning Representations for Counterfactual Inferencecited by 327
  3. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  4. 2018Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networkscited by 278
  5. 2022Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisonscited by 169
  6. 2019Kernel Instrumental Variable Regressioncited by 118
  7. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
  8. 2016Off-policy evaluation for slate recommendationcited by 102
  9. 2020Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertaintycited by 100
  10. 2021Universal Off-Policy Evaluationcited by 43
  11. 2021Semiparametric counterfactual density estimationcited by 42
  12. 2018Counterfactual Mean Embeddingscited by 32
10 more
  1. 2021Off-Policy Risk Assessment in Contextual Banditscited by 29
  2. 2020Off-Policy Evaluation and Learning for External Validity under a Covariate Shiftcited by 26
  3. 2022Conformal Off-Policy Prediction in Contextual Banditscited by 25
  4. 2021Deconditional Downscaling with Gaussian Processescited by 24
  5. 2021BayesIMP: Uncertainty Quantification for Causal Data Fusioncited by 21
  6. 2016Bayesian Learning of Kernel Embeddingscited by 15
  7. 2013Off-policy Learning with Eligibility Traces: A Surveycited by 12
  8. 2020Metrics for Benchmarking and Uncertainty Quantification: Quality, Applicability, and a Path to Best Practices for Machine Learning in Chemistrycited by 9
  9. 2020Bayesian Kernel Two-Sample Testingcited by 3
  10. 2022Sequential Decision Making on Unmatched Data using Bayesian Kernel Embeddingscited by 1

Built on by

Not cited yet.