mlsanityData to 7 Oct 2026

An Entropy Regularization Free Mechanism for Policy-based Reinforcement Learning

arXiv 2106.00707 ↗First posted 1 Jun 2021cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Policy-based reinforcement learning methods suffer from the policy collapse problem. We find valued-based reinforcement learning methods with ε-greedy mechanism are capable of enjoying three characteristics, Closed-form Diversity, Objective-invariant Exploration and Adaptive Trade-off, which help value-based methods avoid the policy collapse problem. However, there does not exist a parallel mechanism for policy-based methods that achieves all three characteristics. In this paper, we propose an entropy regularization free mechanism that is designed for policy-based methods, which achieves Closed-form Diversity, Objective-invariant Exploration and Adaptive Trade-off. Our experiments show that our mechanism is super sample-efficient for policy-based methods and boosts a policy-based baseline to a new State-Of-The-Art on Arcade Learning Environment.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 17 of its 28 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

17 of 28 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  5. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  6. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  7. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  8. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  9. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  10. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 260
  11. 2020Never Give Up: Learning Directed Exploration Strategiescited by 210
  12. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 178
5 more
  1. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 172
  2. 2019V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Controlcited by 85
  3. 2020DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correctioncited by 71
  4. 2019Off-Policy Actor-Critic with Shared Experience Replaycited by 39
  5. 2019The Value Function Polytope in Reinforcement Learningcited by 29

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 2; 2024: 0; 2025: 3.

7 papers.

  1. 2023Learnable Behavior Control: Breaking Atari Human World Records via Sample-Efficient Behavior Selectioncited by 14
  2. 2021A Review for Deep Reinforcement Learning in Atari:Benchmarks, Challenges, and Solutionscited by 12
  3. 2022Generalized Data Distribution Iterationcited by 12
  4. 2025Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewardscited by 11
  5. 2025Fine-tuning Flow Matching Generative Models with Intermediate Feedbackcited by 4
  6. 2023Differentiable Arbitrating in Zero-sum Markov Gamescited by 1
  7. 2025Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruningcited by 1