mlsanityData to 14 Oct 2026

A Max-Min Entropy Framework for Reinforcement Learning

arXiv 2106.10517 ↗First posted 19 Jun 2021cs.LG cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a max-min entropy framework for reinforcement learning (RL) to overcome the limitation of the soft actor-critic (SAC) algorithm implementing the maximum entropy RL in model-free sample-based learning. Whereas the maximum entropy RL guides learning for policies to reach states with high entropy in the future, the proposed max-min entropy framework aims to learn to visit states with low entropy and maximize the entropy of these low-entropy states to promote better exploration. For general Markov decision processes (MDPs), an efficient algorithm is constructed under the proposed max-min entropy framework based on disentanglement of exploration and exploitation. Numerical results show that the proposed algorithm yields drastic performance improvement over the current state-of-the-art RL algorithms.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 32 of its 59 references that are in the map. First posted on arXiv 19 Jun 2021.

Builds on

32 of 59 references.

References · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  7. 2018Exploration by Random Network Distillationcited by 993
  8. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  9. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  10. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  11. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  12. 2013Optimization, Learning, and Games with Predictable Sequencescited by 188
20 more
  1. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 182
  2. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  3. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 173
  4. 2018Learning to Walk via Deep Reinforcement Learningcited by 158
  5. 2019Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methodscited by 148
  6. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 122
  7. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 102
  8. 2017Surprise-Based Intrinsic Motivation for Deep Reinforcement Learningcited by 83
  9. 2018Diversity-Driven Exploration Strategy for Deep Reinforcement Learningcited by 65
  10. 2017Count-Based Exploration in Feature Space for Reinforcement Learningcited by 49
  11. 2019Leveraging exploration in off-policy algorithms via normalizing flowscited by 46
  12. 2017Trust-PCL: An Off-Policy Trust Region Method for Continuous Controlcited by 30
  13. 2018Contingency-Aware Exploration in Reinforcement Learningcited by 26
  14. 2018Generative Adversarial Self-Imitation Learningcited by 25
  15. 2016Combining policy gradient and Q-learningcited by 24
  16. 2020Diversity Actor-Critic: Sample-Aware Entropy Regularization for Sample-Efficient Explorationcited by 20
  17. 2016Frank-Wolfe Algorithms for Saddle Point Problemscited by 18
  18. 2020Solving Non-Convex Non-Differentiable Min-Max Games using Proximal Gradient Methodcited by 10
  19. 2019MULEX: Disentangling Exploitation from Exploration in Deep RLcited by 4
  20. 2019Reward Prediction Error as an Exploration Objective in Deep RLcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 5; 2024: 5; 2025: 11; 2026: 5.

28 papers.

Sort:

By area:

  1. 2024Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulationcited by 20
  2. 2024ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularizationcited by 16
  3. 2022Latent State Marginalization as a Low-cost Approach for Improving Explorationcited by 13
  4. 2023FoX: Formation-aware exploration in multi-agent reinforcement learningcited by 9
  5. 2024Exclusively Penalized Q-learning for Offline Reinforcement Learningcited by 8
  6. 2025The impact of intrinsic rewards on exploration in Reinforcement Learningcited by 8
  7. 2023LESSON: Learning to Integrate Exploration Strategies for Reinforcement Learning via an Option Frameworkcited by 7
  8. 2025When Maximum Entropy Misleads Policy Optimizationcited by 7
  9. 2025Wolfpack Adversarial Attack for Robust Multi-Agent Reinforcement Learningcited by 6
  10. 2025Bridging the Gap in Ophthalmic AI: MM-Retinal-Reason Dataset and OphthaReason Model toward Dynamic Multimodal Reasoningcited by 6
  11. 2021Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learningcited by 5
  12. 2024The Exploration-Exploitation Dilemma Revisited: An Entropy Perspectivecited by 5
16 more
  1. 2025Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learningcited by 5
  2. 2025Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approachcited by 5
  3. 2026Retaining Suboptimal Actions to Follow Shifting Optima in Multi-Agent Reinforcement Learningcited by 3
  4. 2026Flexible Empowerment at Reasoning with Extended Best-of-N Samplingcited by 3
  5. 2023MIMo: A Multi-Modal Infant Model for Studying Cognitive Developmentcited by 2
  6. 2023Predictable Reinforcement Learning Dynamics through Entropy Rate Minimizationcited by 1
  7. 2025Strict Subgoal Execution: Reliable Long-Horizon Planning in Hierarchical Reinforcement Learningcited by 1
  8. 2026Pseudo-Quantized Actor-Critic Algorithm for Robustness to Noisy Temporal Difference Errorcited by 1
  9. 2026Interaction-Breaking Adversarial Learning Framework for Robust Multi-Agent Reinforcement Learningcited by 1
  10. 2023Virtual Action Actor-Critic Framework for Exploration (Student Abstract)cited by 0
  11. 2024Semantic Information G Theory for Range Control with Tradeoff between Purposiveness and Efficiencycited by 0
  12. 2025Meta Policy Switching for Secure UAV Deconfliction in Adversarial Airspacecited by 0
  13. 2025Failure Modes of Maximum Entropy RLHFcited by 0
  14. 2025Complexity-Regularized Proximal Policy Optimizationcited by 0
  15. 2025Dense and Diverse Goal Coverage in Multi Goal Reinforcement Learningcited by 0
  16. 2026Explore Beyond the Boundary Using Entropic Informationcited by 0