Diversity Progress for Goal Selection in Discriminability-Motivated RL
Not on the map. Why?
Abstract (arXiv)
Non-uniform goal selection has the potential to improve the reinforcement learning (RL) of skills over uniform-random selection. In this paper, we introduce a method for learning a goal-selection policy in intrinsically-motivated goal-conditioned RL: "Diversity Progress" (DP). The learner forms a curriculum based on observed improvement in discriminability over its set of goals. Our proposed method is applicable to the class of discriminability-motivated agents, where the intrinsic reward is computed as a function of the agent's certainty of following the true goal being pursued. This reward can motivate the agent to learn a set of diverse skills without extrinsic rewards. We demonstrate empirically that a DP-motivated agent can learn a set of distinguishable skills faster than previous approaches, and do so without suffering from a collapse of the goal distribution -- a known issue with some prior approaches. We end with plans to take this proof-of-concept forward.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 13 of its 27 references that are in the map. First posted on arXiv 3 Nov 2024.
Builds on
13 of 27 references.
References · 13
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
- 2016Variational Intrinsic Controlcited by 262
- 2018Visual Reinforcement Learning with Imagined Goalscited by 248
- 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
- 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
- 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
- 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 83
- 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 78
- 2023Empirical Design in Reinforcement Learningcited by 54
- 2020Relative Variational Intrinsic Controlcited by 34
Built on by
Citing papers per year: 2025: 2.
2 papers.