# Conservative Policy Construction Using Variational Autoencoders for Logged Data with Missing Values

arXiv: https://arxiv.org/abs/2109.03747 · HTML: https://mlsanity.com/paper/2109.03747/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 60 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 18 of 18)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [Handling Incomplete Heterogeneous Data using VAEs](https://mlsanity.com/paper/1807.03653/) (2018)
- [Learning to be Safe: Deep RL with a Safety Critic](https://mlsanity.com/paper/2010.14603/) (2020)
- [Optimal and Adaptive Off-policy Evaluation in Contextual Bandits](https://mlsanity.com/paper/1612.01205/) (2016)
- [Off-policy evaluation for slate recommendation](https://mlsanity.com/paper/1605.04812/) (2016)
- [EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE](https://mlsanity.com/paper/1809.11142/) (2018)
- [Unsupervised Learning of 3D Structure from Images](https://mlsanity.com/paper/1607.00662/) (2016)
- [Conservative Bandits](https://mlsanity.com/paper/1602.04282/) (2016)
- [Causal Inference with Noisy and Missing Covariates via Matrix Factorization](https://mlsanity.com/paper/1806.00811/) (2018)
- [Doubly robust treatment effect estimation with missing attributes](https://mlsanity.com/paper/1910.10624/) (2019)
- [MissDeepCausal: Causal Inference from Incomplete Data Using Deep Latent Variable Models](https://mlsanity.com/paper/2002.10837/) (2020)
- [Global Bandits](https://mlsanity.com/paper/1503.08370/) (2015)
- [Constructing Effective Personalized Policies Using Counterfactual Inference from Biased Data Sets with Many Features](https://mlsanity.com/paper/1612.08082/) (2016)
- [Exploiting Relevance for Online Decision-Making in High-Dimensions](https://mlsanity.com/paper/1907.00783/) (2019)

## Built on by (most cited first, 1 of 1)

- [Estimating counterfactual treatment outcomes over time in complex multiagent scenarios](https://mlsanity.com/paper/2206.01900/) (2022)

## Abstract (arXiv)

In high-stakes applications of data-driven decision making like healthcare, it is of paramount importance to learn a policy that maximizes the reward while avoiding potentially dangerous actions when there is uncertainty. There are two main challenges usually associated with this problem. Firstly, learning through online exploration is not possible due to the critical nature of such applications. Therefore, we need to resort to observational datasets with no counterfactuals. Secondly, such datasets are usually imperfect, additionally cursed with missing values in the attributes of features. In this paper, we consider the problem of constructing personalized policies using logged data when there are missing values in the attributes of features in both training and test data. The goal is to recommend an action (treatment) when Xt, a degraded version of Xb with missing values, is observed. We consider three strategies for dealing with missingness. In particular, we introduce the textitconservative strategy where the policy is designed to safely handle the uncertainty due to missingness. In order to implement this strategy we need to estimate posterior distribution p(Xb|Xt), we use variational autoencoder to achieve this. In particular, our method is based on partial variational autoencoders (PVAE) which are designed to capture the underlying structure of features with missing values.