mlsanityData to 7 Oct 2026

Explicit User Manipulation in Reinforcement Learning Based Recommender Systems

arXiv 2203.10629 ↗First posted 20 Mar 2022cs.LG cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Recommender systems are highly prevalent in the modern world due to their value to both users and platforms and services that employ them. Generally, they can improve the user experience and help to increase satisfaction, but they do not come without risks. One such risk is that of their effect on users and their ability to play an active role in shaping user preferences. This risk is more significant for reinforcement learning based recommender systems. These are capable of learning for instance, how recommended content shown to a user today may tamper that user's preference for other content recommended in the future. Reinforcement learning based recommendation systems can thus implicitly learn to influence users if that means maximizing clicks, engagement, or consumption. On social news and media platforms, in particular, this type of behavior is cause for alarm. Social media undoubtedly plays a role in public opinion and has been shown to be a contributing factor to increased political polarization. Recommender systems on such platforms, therefore, have great potential to influence users in undesirable ways. However, it may also be possible for this form of manipulation to be used intentionally. With advancements in political opinion dynamics modeling and larger collections of user data, explicit user manipulation in which the beliefs and opinions of users are tailored towards a certain end emerges as a significant concern in reinforcement learning based recommender systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 49 references that are in the map. First posted on arXiv 20 Mar 2022.

Builds on

20 of 49 references.

References · 20

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  3. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  4. 2018Distributed Prioritized Experience Replaycited by 260
  5. 2021Reinforcement learning based recommender systems: A surveycited by 187
  6. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 107
  7. 2018Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendationcited by 101
  8. 2019RecSim: A Configurable Simulation Platform for Recommender Systemscited by 91
  9. 2018RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertisingcited by 65
  10. 2018Generative Adversarial User Model for Reinforcement Learning Based Recommendation Systemcited by 64
  11. 2021Partially Observable Markov Decision Processes (POMDPs) and Roboticscited by 49
  12. 2020Hidden Incentives for Auto-Induced Distributional Shiftcited by 42
8 more
  1. 2020News Recommender System: A review of recent progress, challenges, and opportunitiescited by 24
  2. 2021User Tampering in Reinforcement Learning Recommender Systemscited by 23
  3. 2016Optimal Recommendation to Users that React: Online Learning for a Class of POMDPscited by 13
  4. 2016Modeling confirmation bias and polarizationcited by 11
  5. 2016Partially Observable Markov Decision Process for Recommender Systemscited by 11
  6. 2021Designing Recommender Systems to Depolarizecited by 11
  7. 2019Advantage Amplification in Slowly Evolving Latent-State Environmentscited by 4
  8. 2019A Model for the Influence of Media on the Ideology of Content in Online Social Networkscited by 2

Built on by

Not cited yet.