# Evaluating and Learning Robust Bandit Policies Under Uncertain Causal Mechanisms

arXiv: https://arxiv.org/abs/2508.02812 · HTML: https://mlsanity.com/paper/2508.02812/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Builds on (most cited first, 29 of 29)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Is Pessimism Provably Efficient for Offline RL?](https://mlsanity.com/paper/2012.15085/) (2020)
- [Information-Theoretic Considerations in Batch Reinforcement Learning](https://mlsanity.com/paper/1905.00360/) (2019)
- [Causal Discovery from Heterogeneous/Nonstationary Data with Independent Changes](https://mlsanity.com/paper/1903.01672/) (2019)
- [Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions](https://mlsanity.com/paper/1707.06422/) (2017)
- [Invariance, Causality and Robustness](https://mlsanity.com/paper/1812.08233/) (2018)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Causal Bandits: Learning Good Interventions via Causal Inference](https://mlsanity.com/paper/1606.03203/) (2016)
- [DoWhy: An End-to-End Library for Causal Inference](https://mlsanity.com/paper/2011.04216/) (2020)
- [External Validity: From Do-Calculus to Transportability Across Populations](https://mlsanity.com/paper/1503.01603/) (2015)
- [A causal framework for distribution generalization](https://mlsanity.com/paper/2006.07433/) (2020)
- [DoWhy-GCM: An extension of DoWhy for causal inference in graphical causal models](https://mlsanity.com/paper/2206.06821/) (2022)
- [Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning](https://mlsanity.com/paper/2202.09667/) (2022)
- [Modeling the Data-Generating Process is Necessary for Out-of-Distribution Generalization](https://mlsanity.com/paper/2206.07837/) (2022)
- [Distributionally Robust Counterfactual Risk Minimization](https://mlsanity.com/paper/1906.06211/) (2019)
- [Distributionally Robust Batch Contextual Bandits](https://mlsanity.com/paper/2006.05630/) (2020)
- [Causal Bandits for Linear Structural Equation Models](https://mlsanity.com/paper/2208.12764/) (2022)
- [Invariant Policy Learning: A Causal Perspective](https://mlsanity.com/paper/2106.00808/) (2021)
- [Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning](https://mlsanity.com/paper/2405.14335/) (2024)
- [Offline Contextual Bandits with Overparameterized Models](https://mlsanity.com/paper/2006.15368/) (2020)
- [Evaluating Robustness to Dataset Shift via Parametric Robustness Sets](https://mlsanity.com/paper/2205.15947/) (2022)
- [Wasserstein Distributionally Robust Policy Evaluation and Learning for Contextual Bandits](https://mlsanity.com/paper/2309.08748/) (2023)
- [Policy Learning under Biased Sample Selection](https://mlsanity.com/paper/2304.11735/) (2023)
- [Distributionally Robust Policy Learning under Concept Drifts](https://mlsanity.com/paper/2412.14297/) (2024)
- [Improving Offline Contextual Bandits with Distributional Robustness](https://mlsanity.com/paper/2011.06835/) (2020)
- [Data mining for censored time-to-event data: A Bayesian network model for predicting cardiovascular risk from electronic health record data](https://mlsanity.com/paper/1404.2189/) (2014)
- [Robust Causal Bandits for Linear Models](https://mlsanity.com/paper/2310.19794/) (2023)
- [Self-concordant inclusions: A unified framework for path-following generalized Newton-type algorithms](https://mlsanity.com/paper/1707.07403/) (2017)
- [Improved Bound for Robust Causal Bandits with Linear Models](https://mlsanity.com/paper/2405.07795/) (2024)

## Abstract (arXiv)

Causal graphical models can encode large amounts structural knowledge, both from the background knowledge of domain experts and the structural knowledge discovered from randomized experiments or observational data. However, though we may know the general structure of causal relationships, we often do not know the exact causal mechanisms. In this work, we propose a causal multi-armed bandit evaluation and learning algorithm that can reason effectively despite uncertainty over conditional probability distributions. Further, we show how conditional independence testing can be used to choose variables for modeling. We find that the structural equation model (SEM) approach gives more accurate evaluations compared to traditional approaches, particularly as the range of possible causal mechanisms grows. Further, the SEM approach learns low-variance policies, and it learns an optimal policy, assuming the model is sufficiently well-specified. Traditional approaches can converge to local extrema or fail to converge at all.