# Online Semi-Supervised Learning in Contextual Bandits with Episodic Reward

arXiv: https://arxiv.org/abs/2009.08457 · HTML: https://mlsanity.com/paper/2009.08457/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2020.

## Built on by (most cited first, 5 of 5)

- [Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlook](https://mlsanity.com/paper/2210.13623/) (2022)
- [SupervisorBot: NLP-Annotated Real-Time Recommendations of Psychotherapy Treatment Strategies with Deep Reinforcement Learning](https://mlsanity.com/paper/2208.13077/) (2022)
- [Optimal Epidemic Control as a Contextual Combinatorial Bandit with Budget](https://mlsanity.com/paper/2106.15808/) (2021)
- [Evolutionary Multi-Armed Bandits with Genetic Thompson Sampling](https://mlsanity.com/paper/2205.10113/) (2022)
- [A Reinforcement Learning Framework for Online Speaker Diarization](https://mlsanity.com/paper/2302.10924/) (2023)

## Abstract (arXiv)

We considered a novel practical problem of online learning with episodically revealed rewards, motivated by several real-world applications, where the contexts are nonstationary over different episodes and the reward feedbacks are not always available to the decision making agents. For this online semi-supervised learning setting, we introduced Background Episodic Reward LinUCB (BerlinUCB), a solution that easily incorporates clustering as a self-supervision module to provide useful side information when rewards are not observed. Our experiments on a variety of datasets, both in stationary and nonstationary environments of six different scenarios, demonstrated clear advantages of the proposed approach over the standard contextual bandit. Lastly, we introduced a relevant real-life example where this problem setting is especially useful.