# Diversified Recommendations for Agents with Adaptive Preferences

arXiv: https://arxiv.org/abs/2210.07773 · HTML: https://mlsanity.com/paper/2210.07773/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 20 Sep 2022.

## Builds on (most cited first, 22 of 22)

- [Introduction to Online Convex Optimization](https://mlsanity.com/paper/1909.05207/) (2019)
- [Performative Prediction](https://mlsanity.com/paper/2002.06673/) (2020)
- [Strategic Classification from Revealed Preferences](https://mlsanity.com/paper/1710.07887/) (2017)
- [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](https://mlsanity.com/paper/1710.11214/) (2017)
- [Preference-based Online Learning with Dueling Bandits: A Survey](https://mlsanity.com/paper/1807.11398/) (2018)
- [Stochastic Optimization for Performative Prediction](https://mlsanity.com/paper/2006.06887/) (2020)
- [Feedback Loop and Bias Amplification in Recommender Systems](https://mlsanity.com/paper/2007.13019/) (2020)
- [Who Leads and Who Follows in Strategic Classification?](https://mlsanity.com/paper/2106.12529/) (2021)
- [Strategizing against No-regret Learners](https://mlsanity.com/paper/1909.13861/) (2019)
- [Selling to a No-Regret Buyer](https://mlsanity.com/paper/1711.09176/) (2017)
- [Understanding Echo Chambers in E-commerce Recommender Systems](https://mlsanity.com/paper/2007.02474/) (2020)
- [Regret Minimization with Performative Feedback](https://mlsanity.com/paper/2202.00628/) (2022)
- [Optimizing Long-term Social Welfare in Recommender Systems: A Constrained Matching Approach](https://mlsanity.com/paper/2008.00104/) (2020)
- [Learning in Auctions: Regret is Hard, Envy is Easy](https://mlsanity.com/paper/1511.01411/) (2015)
- [Online learning over a finite action set with limited switching](https://mlsanity.com/paper/1803.01548/) (2018)
- [Chasing Nested Convex Bodies Nearly Optimally](https://mlsanity.com/paper/1811.00999/) (2018)
- [Watch and Learn: Optimizing from Revealed Preferences Feedback](https://mlsanity.com/paper/1504.01033/) (2015)
- [Deconvolving Feedback Loops in Recommender Systems](https://mlsanity.com/paper/1703.01049/) (2017)
- [A Robust Sparse Fourier Transform in the Continuous Setting](https://mlsanity.com/paper/1609.00896/) (2016)
- [Trust and Trustworthiness in Social Recommender Systems](https://mlsanity.com/paper/1903.01780/) (2019)
- [Sequential Choice Bandits with Feedback for Personalizing users' experience](https://mlsanity.com/paper/2101.01572/) (2021)
- [Emergent Instabilities in Algorithmic Feedback Loops](https://mlsanity.com/paper/2201.07203/) (2022)

## Built on by (most cited first, 9 of 9)

- [Performative Recommendation: Diversifying Content via Strategic Incentives](https://mlsanity.com/paper/2302.04336/) (2023)
- [Accounting for AI and Users Shaping One Another: The Role of Mathematical Models](https://mlsanity.com/paper/2404.12366/) (2024)
- [AI-Assisted Decision Making with Human Learning](https://mlsanity.com/paper/2502.13062/) (2025)
- [Online Recommendations for Agents with Discounted Adaptive Preferences](https://mlsanity.com/paper/2302.06014/) (2023)
- [AURO: Reinforcement Learning for Adaptive User Retention Optimization in Recommender Systems](https://mlsanity.com/paper/2310.03984/) (2023)
- [When Are Two Lists Better than One?: Benefits and Harms in Joint Decision-making](https://mlsanity.com/paper/2308.11721/) (2023)
- [On the Impact of Performative Risk Minimization for Binary Random Variables](https://mlsanity.com/paper/2502.02331/) (2025)
- [Multi-Round Human-AI Collaboration with User-Specified Requirements](https://mlsanity.com/paper/2602.17646/) (2026)
- [Online Stackelberg Optimization via Nonlinear Control](https://mlsanity.com/paper/2406.18805/) (2024)

## Abstract (arXiv)

When an Agent visits a platform recommending a menu of content to select from, their choice of item depends not only on fixed preferences, but also on their prior engagements with the platform. The Recommender's primary objective is typically to encourage content consumption which optimizes some reward, such as ad revenue, but they often also aim to ensure that a wide variety of content is consumed by the Agent over time. We formalize this problem as an adversarial bandit task. At each step, the Recommender presents a menu of k (out of n) items to the Agent, who selects one item in the menu according to their unknown preference model, which maps their history of past items to relative selection probabilities. The Recommender then observes the Agent's chosen item and receives bandit feedback of the item's reward. In addition to optimizing reward from selected items, the Recommender must also ensure that the total distribution of chosen items has sufficiently high entropy. We define a class of preference models which are locally learnable, i.e. behavior over the entire domain can be estimated by only observing behavior in a small region; this includes models representable by bounded-degree polynomials as well as functions with a sparse Fourier basis. For this class, we give an algorithm for the Recommender which obtains tildeO(T3/4) regret against all item distributions satisfying two conditions: they are sufficiently diversified, and they are instantaneously realizable at any history by some distribution over menus. We show that these conditions are closely connected: all sufficiently high-entropy distributions are instantaneously realizable at any item history. We also give a set of negative results justifying our assumptions, in the form of a runtime lower bound for non-local learning and linear regret lower bounds for alternate benchmarks.