# Contextual Bandits with Budgeted Information Reveal

arXiv: https://arxiv.org/abs/2305.18511 · HTML: https://mlsanity.com/paper/2305.18511/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 60 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 11 of 11)

- [Bandits with Knapsacks](https://mlsanity.com/paper/1305.2545/) (2013)
- [Online Learning under Delayed Feedback](https://mlsanity.com/paper/1306.0686/) (2013)
- [Bandits with concave rewards and convex knapsacks](https://mlsanity.com/paper/1402.5758/) (2014)
- [Linear Contextual Bandits with Knapsacks](https://mlsanity.com/paper/1507.06738/) (2015)
- [Stochastic Bandits with Linear Constraints](https://mlsanity.com/paper/2006.10185/) (2020)
- [Linear Bandits with Stochastic Delayed Feedback](https://mlsanity.com/paper/1807.02089/) (2018)
- [Resourceful Contextual Bandits](https://mlsanity.com/paper/1402.6779/) (2014)
- [An efficient algorithm for contextual bandits with knapsacks, and an extension to concave objectives](https://mlsanity.com/paper/1506.03374/) (2015)
- [Algorithms with Logarithmic or Sublinear Regret for Constrained Contextual Bandits](https://mlsanity.com/paper/1504.06937/) (2015)
- [Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-implementation Guidelines](https://mlsanity.com/paper/2206.03944/) (2022)
- [Inventory Balancing with Online Learning](https://mlsanity.com/paper/1810.05640/) (2018)

## Built on by (most cited first, 3 of 3)

- [Automated Off-Policy Estimator Selection via Supervised Learning](https://mlsanity.com/paper/2406.18022/) (2024)
- [HR-Bandit: Human-AI Collaborated Linear Recourse Bandit](https://mlsanity.com/paper/2410.14640/) (2024)
- [LIBRA: Language Model Informed Bandit Recourse Algorithm for Personalized Treatment Planning](https://mlsanity.com/paper/2601.11905/) (2026)

## Abstract (arXiv)

Contextual bandit algorithms are commonly used in digital health to recommend personalized treatments. However, to ensure the effectiveness of the treatments, patients are often requested to take actions that have no immediate benefit to them, which we refer to as pro-treatment actions. In practice, clinicians have a limited budget to encourage patients to take these actions and collect additional information. We introduce a novel optimization and learning algorithm to address this problem. This algorithm effectively combines the strengths of two algorithmic approaches in a seamless manner, including 1) an online primal-dual algorithm for deciding the optimal timing to reach out to patients, and 2) a contextual bandit learning algorithm to deliver personalized treatment to the patient. We prove that this algorithm admits a sub-linear regret bound. We illustrate the usefulness of this algorithm on both synthetic and real-world data.