# Sequential Decision Problems with Weak Feedback

arXiv: https://arxiv.org/abs/2212.11603 · HTML: https://mlsanity.com/paper/2212.11603/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 123 references that are in the map.
- First posted on arXiv 22 Dec 2022.

## Builds on (most cited first, 16 of 16)

- [Thompson Sampling for Combinatorial Semi-Bandits](https://mlsanity.com/paper/1803.04623/) (2018)
- [Combinatorial Multi-Armed Bandit with General Reward Functions](https://mlsanity.com/paper/1610.06603/) (2016)
- [SIC-MMAB: Synchronisation Involves Communication in Multiplayer Multi-Armed Bandits](https://mlsanity.com/paper/1809.08151/) (2018)
- [Online learning in repeated auctions](https://mlsanity.com/paper/1511.05720/) (2015)
- [Provably Optimal Algorithms for Generalized Linear Contextual Bandits](https://mlsanity.com/paper/1703.00048/) (2017)
- [Statistical Efficiency of Thompson Sampling for Combinatorial Semi-Bandits](https://mlsanity.com/paper/2006.06613/) (2020)
- [From Bandits to Experts: A Tale of Domination and Independence](https://mlsanity.com/paper/1307.4564/) (2013)
- [Online Learning with Gaussian Payoffs and Side Observations](https://mlsanity.com/paper/1510.08108/) (2015)
- [An adaptive stochastic optimization algorithm for resource allocation](https://mlsanity.com/paper/1902.04376/) (2019)
- [Multi-Player Bandits: A Trekking Approach](https://mlsanity.com/paper/1809.06040/) (2018)
- [Online Algorithm for Unsupervised Sequential Selection with Contextual Information](https://mlsanity.com/paper/2010.12353/) (2020)
- [Censored Semi-Bandits for Resource Allocation](https://mlsanity.com/paper/2104.05781/) (2021)
- [Combinatorial Bandits Revisited](https://mlsanity.com/paper/1502.03475/) (2015)
- [Multi-Armed Bandits with Censored Consumption of Resources](https://mlsanity.com/paper/2011.00813/) (2020)
- [A Minimax Optimal Algorithm for Crowdsourcing](https://mlsanity.com/paper/1606.00226/) (2016)
- [Multi-Player Bandits Revisited](https://mlsanity.com/paper/1711.02317/) (2017)

## Abstract (arXiv)

This thesis considers sequential decision problems, where the loss/reward incurred by selecting an action may not be inferred from observed feedback. A major part of this thesis focuses on the unsupervised sequential selection problem, where one can not infer the loss incurred for selecting an action from observed feedback. We also introduce a new setup named Censored Semi Bandits, where the loss incurred for selecting an action can be observed under certain conditions. Finally, we study the channel selection problem in the communication networks, where the reward for an action is only observed when no other player selects that action to play in the round. These problems find applications in many fields like healthcare, crowd-sourcing, security, adaptive resource allocation, among many others. This thesis aims to address the above-described sequential decision problems by exploiting specific structures these problems exhibit. We develop provably optimal algorithms for each of these setups with weak feedback and validate their empirical performance on different problem instances derived from synthetic and real datasets.