mlsanityData to 14 Oct 2026

Increasing Entropy to Boost Policy Gradient Performance on Personalization Tasks

arXiv 2310.05324 ↗First posted 9 Oct 2023cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this effort, we consider the impact of regularization on the diversity of actions taken by policies generated from reinforcement learning agents trained using a policy gradient. Policy gradient agents are prone to entropy collapse, which means certain actions are seldomly, if ever, selected. We augment the optimization objective function for the policy with terms constructed from various varphi-divergences and Maximum Mean Discrepancy which encourages current policies to follow different state visitation and/or action choice distribution than previously computed policies. We provide numerical experiments using MNIST, CIFAR10, and Spotify datasets. The results demonstrate the advantage of diversity-promoting policy regularization and that its use on gradient-based approaches have significantly improved performance on a variety of personalization tasks. Furthermore, numerical evidence is given to show that policy regularization increases performance without losing accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 51 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

19 of 51 references.

References · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2018Demystifying MMD GANscited by 1,293
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  7. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  8. 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 254
  9. 2018Understanding the impact of entropy on policy optimizationcited by 176
  10. 2018Diversity-Driven Exploration Strategy for Deep Reinforcement Learningcited by 65
  11. 2019Information asymmetry in KL-regularized RLcited by 53
  12. 2019Diversity-Inducing Policy Gradient: Using Maximum Mean Discrepancy to Find a Set of Diverse Policiescited by 34
7 more
  1. 2015Counterfactual Risk Minimization: Learning from Logged Bandit Feedbackcited by 33
  2. 2013Exploration in Interactive Personalized Music Recommendation: A Reinforcement Learning Approachcited by 22
  3. 2017An Actor-Critic Contextual Bandit Algorithm for Personalized Mobile Health Interventionscited by 17
  4. 2018Robust Actor-Critic Contextual Bandit for Mobile Health (mHealth) Interventionscited by 9
  5. 2018A clustering-based reinforcement learning approach for tailored personalization of e-Health interventionscited by 8
  6. 2022Examining Policy Entropy of Reinforcement Learning Agents for Personalization Taskscited by 3
  7. 2020An adaptive stochastic gradient-free approach for high-dimensional blackbox optimizationcited by 2

Built on by

Not cited yet.