# Influencing Bandits: Arm Selection for Preference Shaping

arXiv: https://arxiv.org/abs/2403.00036 · HTML: https://mlsanity.com/paper/2403.00036/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 32 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Builds on (most cited first, 8 of 8)

- [Introduction to Multi-Armed Bandits](https://mlsanity.com/paper/1904.07272/) (2019)
- [Rotting Bandits](https://mlsanity.com/paper/1702.07274/) (2017)
- [Learning Contextual Bandits in a Non-stationary Environment](https://mlsanity.com/paper/1805.09365/) (2018)
- [Optimal Recommendation to Users that React: Online Learning for a Class of POMDPs](https://mlsanity.com/paper/1603.09233/) (2016)
- [On the Whittle Index for Restless Multi-armed Hidden Markov Bandits](https://mlsanity.com/paper/1603.04739/) (2016)
- [Bandit Learning with Positive Externalities](https://mlsanity.com/paper/1802.05693/) (2018)
- [Multi-Armed Bandits for Correlated Markovian Environments with Smoothed Reward Feedback](https://mlsanity.com/paper/1803.04008/) (2018)
- [Spread, then Target, and Advertise in Waves: Optimal Budget Allocation Across Advertising Channels](https://mlsanity.com/paper/1702.03432/) (2017)

## Abstract (arXiv)

We consider a non stationary multi-armed bandit in which the population preferences are positively and negatively reinforced by the observed rewards. The objective of the algorithm is to shape the population preferences to maximize the fraction of the population favouring a predetermined arm. For the case of binary opinions, two types of opinion dynamics are considered -- decreasing elasticity (modeled as a Polya urn with increasing number of balls) and constant elasticity (using the voter model). For the first case, we describe an Explore-then-commit policy and a Thompson sampling policy and analyse the regret for each of these policies. We then show that these algorithms and their analyses carry over to the constant elasticity case. We also describe a Thompson sampling based algorithm for the case when more than two types of opinions are present. Finally, we discuss the case where presence of multiple recommendation systems gives rise to a trade-off between their popularity and opinion shaping objectives.