# Robust Counterfactual Policy Optimisation via Nondeterministic Causal Models

arXiv: https://arxiv.org/abs/2608.02893 · HTML: https://mlsanity.com/paper/2608.02893/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Builds on (most cited first, 11 of 11)

- [Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models](https://mlsanity.com/paper/1905.05824/) (2019)
- [Off-policy Policy Evaluation For Sequential Decisions Under Unobserved Confounding](https://mlsanity.com/paper/2003.05623/) (2020)
- [Counterfactually Guided Off-policy Transfer in Clinical Settings](https://mlsanity.com/paper/2006.11654/) (2020)
- [Robust Fitted-Q-Evaluation and Iteration under Sequentially Exogenous Unobserved Confounders](https://mlsanity.com/paper/2302.00662/) (2023)
- [Finding Counterfactually Optimal Action Sequences in Continuous State Spaces](https://mlsanity.com/paper/2306.03929/) (2023)
- [Model-Free and Model-Based Policy Evaluation when Causality is Uncertain](https://mlsanity.com/paper/2204.00956/) (2022)
- [Offline Policy Evaluation and Optimization under Confounding](https://mlsanity.com/paper/2211.16583/) (2022)
- [Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes](https://mlsanity.com/paper/2404.00099/) (2024)
- [Nondeterministic Causal Models](https://mlsanity.com/paper/2405.14001/) (2024)
- [Large Language Models as Nondeterministic Causal Models](https://mlsanity.com/paper/2509.22297/) (2025)
- [Causal Counterfactuals Reconsidered](https://mlsanity.com/paper/2512.12804/) (2025)

## Abstract (arXiv)

Counterfactual inference approaches for sequential decision-making typically assume deterministic causal models, where all randomness stems from latent variables. However, Markov Decision Processes (MDPs) are inherently stochastic. We address this by formalising counterfactual policy optimisation under probabilistic nondeterministic causal models, which properly separates latent confounding from irreducible stochasticity, and here propose a first practical optimisation problem for identifying robust counterfactual policies under a sensitivity analysis framework. We validate our approach on a sepsis treatment simulator, where diabetes status acts as a hidden global confounder.