mlsanityData to 7 Oct 2026

Maximum Entropy-Regularized Multi-Goal Reinforcement Learning

arXiv 1905.08786 ↗First posted 21 May 2019cs.LG stat.MLcited by 39

Not on the map. Why?

Abstract (arXiv)

In Multi-Goal Reinforcement Learning, an agent learns to achieve multiple goals with a goal-conditioned policy. During learning, the agent first collects the trajectories into a replay buffer, and later these trajectories are selected randomly for replay. However, the achieved goals in the replay buffer are often biased towards the behavior policies. From a Bayesian perspective, when there is no prior knowledge about the target goal distribution, the agent should learn uniformly from diverse achieved goals. Therefore, we first propose a novel multi-goal RL objective based on weighted entropy. This objective encourages the agent to maximize the expected return, as well as to achieve more diverse goals. Secondly, we developed a maximum entropy-based prioritization framework to optimize the proposed objective. For evaluation of this framework, we combine it with Deep Deterministic Policy Gradient, both with or without Hindsight Experience Replay. On a set of multi-goal robotic tasks of OpenAI Gym, we compare our method with other baselines and show promising improvements in both performance and sample-efficiency.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 8; 2023: 6; 2024: 2; 2025: 9; 2026: 1.

39 papers.

Sort:

By area:

  1. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  2. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  3. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  4. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 71
  5. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 69
  6. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  7. 2023What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?cited by 36
  8. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  9. 2021Efficient Continuous Control with Double Actors and Regularized Criticscited by 20
  10. 2021Mutual Information State Intrinsic Controlcited by 18
  11. 2022Phasic Self-Imitative Reduction for Sparse-Reward Goal-Conditioned Reinforcement Learningcited by 16
  12. 2024ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularizationcited by 16
27 more
  1. 2021MHER: Model-based Hindsight Experience Replaycited by 14
  2. 2025EntroPIC: Towards Stable Long-Term Training of LLMs via Entropy Stabilization with Proportional-Integral Controlcited by 13
  3. 2021HYPER: Learned Hybrid Trajectory Prediction via Factored Inference and Adaptive Samplingcited by 12
  4. 2022Beyond Supervised Continual Learning: a Reviewcited by 11
  5. 2025Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learningcited by 10
  6. 2021DisTop: Discovering a Topological representation to learn diverse and rewarding skillscited by 7
  7. 2021Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learningcited by 7
  8. 2021Hindsight Goal Ranking on Replay Buffer for Sparse Reward Environmentcited by 7
  9. 2021Bias-reduced Multi-step Hindsight Experience Replay for Efficient Multi-goal Reinforcement Learningcited by 4
  10. 2021Adaptive Multi-Goal Explorationcited by 4
  11. 2025Improving Environment Novelty Quantification for Effective Unsupervised Environment Designcited by 4
  12. 2025Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learningcited by 4
  13. 2025LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Modelscited by 4
  14. 2025Performative Policy Gradient: Optimality in Performative Reinforcement Learningcited by 4
  15. 2024Evolution Guided Generative Flow Networkscited by 3
  16. 2023L-SA: Learning Under-Explored Targets in Multi-Target Reinforcement Learningcited by 2
  17. 2025Constrained Sliced Wasserstein Embeddingcited by 2
  18. 2021Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learningcited by 1
  19. 2023Differentiable Arbitrating in Zero-sum Markov Gamescited by 1
  20. 2023Predictable Reinforcement Learning Dynamics through Entropy Rate Minimizationcited by 1
  21. 2023Contact Energy Based Hindsight Experience Prioritizationcited by 1
  22. 2023Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularizationcited by 1
  23. 2022A Fully Controllable Agent in the Path Planning using Goal-Conditioned Reinforcement Learningcited by 0
  24. 2022SHIRO: Soft Hierarchical Reinforcement Learningcited by 0
  25. 2025Complexity-Regularized Proximal Policy Optimizationcited by 0
  26. 2025Dense and Diverse Goal Coverage in Multi Goal Reinforcement Learningcited by 0
  27. 2026Entropy-Regularized Reinforcement Learning for Linear-Quadratic Stackelberg Differential Games in Regime-Switching Diffusion Modelscited by 0