# Sequential Decision Making on Unmatched Data using Bayesian Kernel Embeddings

arXiv: https://arxiv.org/abs/2210.13692 · HTML: https://mlsanity.com/paper/2210.13692/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [On Kernelized Multi-armed Bandits](https://mlsanity.com/paper/1704.00445/) (2017)
- [A Survey on Contextual Multi-armed Bandits](https://mlsanity.com/paper/1508.03326/) (2015)
- [Best Arm Identification for Contaminated Bandits](https://mlsanity.com/paper/1802.09514/) (2018)
- [Deconditional Downscaling with Gaussian Processes](https://mlsanity.com/paper/2105.12909/) (2021)
- [BayesIMP: Uncertainty Quantification for Causal Data Fusion](https://mlsanity.com/paper/2106.03477/) (2021)
- [Bayesian optimisation under uncertain inputs](https://mlsanity.com/paper/1902.07908/) (2019)
- [Deep Contextual Multi-armed Bandits](https://mlsanity.com/paper/1807.09809/) (2018)
- [Bayesian Learning of Kernel Embeddings](https://mlsanity.com/paper/1603.02160/) (2016)
- [Kernel Mean Estimation and Stein's Effect](https://mlsanity.com/paper/1306.0842/) (2013)

## Built on by (most cited first, 1 of 1)

- [Bayesian Counterfactual Mean Embeddings and Off-Policy Evaluation](https://mlsanity.com/paper/2211.01518/) (2022)

## Abstract (arXiv)

The problem of sequentially maximizing the expectation of a function seeks to maximize the expected value of a function of interest without having direct control on its features. Instead, the distribution of such features depends on a given context and an action taken by an agent. In contrast to Bayesian optimization, the arguments of the function are not under agent's control, but are indirectly determined by the agent's action based on a given context. If the information of the features is to be included in the maximization problem, the full conditional distribution of such features, rather than its expectation only, needs to be accounted for. Furthermore, the function is itself unknown, only counting with noisy observations of such function, and potentially requiring the use of unmatched data sets. We propose a novel algorithm for the aforementioned problem which takes into consideration the uncertainty derived from the estimation of both the conditional distribution of the features and the unknown function, by modeling the former as a Bayesian conditional mean embedding and the latter as a Gaussian process. Our algorithm empirically outperforms the current state-of-the-art algorithm in the experiments conducted.