mlsanityData to 7 Oct 2026

Wasserstein Diversity-Enriched Regularizer for Hierarchical Reinforcement Learning

arXiv 2308.00989 ↗First posted 2 Aug 2023cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Hierarchical reinforcement learning composites subpolicies in different hierarchies to accomplish complex tasks.Automated subpolicies discovery, which does not depend on domain knowledge, is a promising approach to generating subpolicies.However, the degradation problem is a challenge that existing methods can hardly deal with due to the lack of consideration of diversity or the employment of weak regularizers. In this paper, we propose a novel task-agnostic regularizer called the Wasserstein Diversity-Enriched Regularizer (WDER), which enlarges the diversity of subpolicies by maximizing the Wasserstein distances among action distributions. The proposed WDER can be easily incorporated into the loss function of existing methods to boost their performance further.Experimental results demonstrate that our WDER improves performance and sample efficiency in comparison with prior work without modifying hyperparameters, which indicates the applicability and robustness of the WDER.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 2 Aug 2023.

Builds on

22 of 34 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  5. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  6. 2016The Option-Critic Architecturecited by 433
  7. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  8. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  9. 2020Primal Wasserstein Imitation Learningcited by 120
  10. 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 107
  11. 2017Meta Learning Shared Hierarchiescited by 93
  12. 2019Wasserstein Dependency Measure for Representation Learningcited by 69
10 more
  1. 2018Latent Space Policies for Hierarchical Reinforcement Learningcited by 66
  2. 2019Sliced Wasserstein Generative Modelscited by 61
  3. 2019DAC: The Double Actor-Critic Architecture for Learning Optionscited by 43
  4. 2019Learning to Score Behaviors for Guided Policy Optimizationcited by 34
  5. 2018Policy Optimization as Wasserstein Gradient Flowscited by 33
  6. 2019Orthogonal Estimation of Wasserstein Distancescited by 32
  7. 2020Efficient Wasserstein Natural Gradients for Reinforcement Learningcited by 23
  8. 2022Hierarchical Sliced Wasserstein Distancecited by 23
  9. 2021Wasserstein Unsupervised Reinforcement Learningcited by 21
  10. 2020Diversity-Enriched Option-Criticcited by 7

Built on by

Not cited yet.