# Contextual Bandit with Herding Effects: Algorithms and Recommendation Applications

arXiv: https://arxiv.org/abs/2408.14432 · HTML: https://mlsanity.com/paper/2408.14432/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 33 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Builds on (most cited first, 8 of 8)

- [Learning to Optimize Via Posterior Sampling](https://mlsanity.com/paper/1301.2609/) (2013)
- [Conversational Contextual Bandit: Algorithm and Application](https://mlsanity.com/paper/1906.01219/) (2019)
- [Contextual Bandits with Latent Confounders: An NMF Approach](https://mlsanity.com/paper/1606.00119/) (2016)
- [Bandits with Partially Observable Confounded Data](https://mlsanity.com/paper/2006.06731/) (2020)
- [Impatient Bandits: Optimizing Recommendations for the Long-Term Without Delay](https://mlsanity.com/paper/2307.09943/) (2023)
- [Efficient Explorative Key-term Selection Strategies for Conversational Contextual Bandits](https://mlsanity.com/paper/2303.00315/) (2023)
- [Instrument-Armed Bandits](https://mlsanity.com/paper/1705.07377/) (2017)
- [Contextual Bandits for adapting to changing User preferences over time](https://mlsanity.com/paper/2009.10073/) (2020)

## Built on by (most cited first, 1 of 1)

- [Scalable and Interpretable Contextual Bandits: A Literature Review and Retail Offer Prototype](https://mlsanity.com/paper/2505.16918/) (2025)

## Abstract (arXiv)

Contextual bandits serve as a fundamental algorithmic framework for optimizing recommendation decisions online. Though extensive attention has been paid to tailoring contextual bandits for recommendation applications, the "herding effects" in user feedback have been ignored. These herding effects bias user feedback toward historical ratings, breaking down the assumption of unbiased feedback inherent in contextual bandits. This paper develops a novel variant of the contextual bandit that is tailored to address the feedback bias caused by the herding effects. A user feedback model is formulated to capture this feedback bias. We design the TS-Conf (Thompson Sampling under Conformity) algorithm, which employs posterior sampling to balance the exploration and exploitation tradeoff. We prove an upper bound for the regret of the algorithm, revealing the impact of herding effects on learning speed. Extensive experiments on datasets demonstrate that TS-Conf outperforms four benchmark algorithms. Analysis reveals that TS-Conf effectively mitigates the negative impact of herding effects, resulting in faster learning and improved recommendation accuracy.