# Thompson Sampling for Contextual Bandit Problems with Auxiliary Safety Constraints

arXiv: https://arxiv.org/abs/1911.00638 · HTML: https://mlsanity.com/paper/1911.00638/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2019.

## Built on by (most cited first, 10 of 10)

- [Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement](https://mlsanity.com/paper/2105.08195/) (2021)
- [Looper: An end-to-end ML platform for product decisions](https://mlsanity.com/paper/2110.07554/) (2021)
- [Stochastic Conservative Contextual Linear Bandits](https://mlsanity.com/paper/2203.15629/) (2022)
- [Constrained Policy Optimization for Controlled Self-Learning in Conversational AI Systems](https://mlsanity.com/paper/2209.08429/) (2022)
- [Experimenting, Fast and Slow: Bayesian Optimization of Long-term Outcomes with Online Experiments](https://mlsanity.com/paper/2506.18744/) (2025)
- [Risk-Aware Continuous Control with Neural Contextual Bandits](https://mlsanity.com/paper/2312.09961/) (2023)
- [BanditLP: Large-Scale Stochastic Optimization for Personalized Recommendations](https://mlsanity.com/paper/2601.15552/) (2026)
- [AI in Pharma for Personalized Sequential Decision-Making: Methods, Applications and Opportunities](https://mlsanity.com/paper/2311.18725/) (2023)
- [Multi-armed Bandits with Missing Outcome](https://mlsanity.com/paper/2411.05661/) (2024)
- [Multi-Task Representation Learning for Conservative Linear Bandits](https://mlsanity.com/paper/2605.12176/) (2026)

## Abstract (arXiv)

Recent advances in contextual bandit optimization and reinforcement learning have garnered interest in applying these methods to real-world sequential decision making problems. Real-world applications frequently have constraints with respect to a currently deployed policy. Many of the existing constraint-aware algorithms consider problems with a single objective (the reward) and a constraint on the reward with respect to a baseline policy. However, many important applications involve multiple competing objectives and auxiliary constraints. In this paper, we propose a novel Thompson sampling algorithm for multi-outcome contextual bandit problems with auxiliary constraints. We empirically evaluate our algorithm on a synthetic problem. Lastly, we apply our method to a real world video transcoding problem and provide a practical way for navigating the trade-off between safety and performance using Bayesian optimization.