mlsanityData to 14 Oct 2026

Stein Variational Goal Generation for adaptive Exploration in Multi-Goal Reinforcement Learning

arXiv 2206.06719 ↗First posted 14 Jun 2022cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

In multi-goal Reinforcement Learning, an agent can share experience between related training tasks, resulting in better generalization for new tasks at test time. However, when the goal space has discontinuities and the reward is sparse, a majority of goals are difficult to reach. In this context, a curriculum over goals helps agents learn by adapting training tasks to their current capabilities. In this work we propose Stein Variational Goal Generation (SVGG), which samples goals of intermediate difficulty for the agent, by leveraging a learned predictive model of its goal reaching capabilities. The distribution of goals is modeled with particles that are attracted in areas of appropriate difficulty using Stein Variational Gradient Descent. We show that SVGG outperforms state-of-the-art multi-goal Reinforcement Learning methods in terms of success coverage in hard exploration problems, and demonstrate that it is endowed with a useful recovery property when the environment changes.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 27 of its 39 references that are in the map. First posted on arXiv 14 Jun 2022.

Builds on

27 of 39 references.

References · 27

  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  2. 2017Hindsight Experience Replaycited by 1,136
  3. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  4. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
  5. 2019Self-Supervised Exploration via Disagreementcited by 265
  6. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  7. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  8. 2018Visual Reinforcement Learning with Imagined Goalscited by 251
  9. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  10. 2021Reinforcement Learning with Prototypical Representationscited by 179
  11. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  12. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
15 more
  1. 2017Stein Variational Gradient Descent as Gradient Flowcited by 152
  2. 2016A Kernel Test of Goodness of Fitcited by 143
  3. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
  4. 2017Measuring Sample Quality with Kernelscited by 119
  5. 2014Generalization and Exploration via Randomized Value Functionscited by 112
  6. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  7. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
  8. 2020Automatic Curriculum Learning through Value Disagreementcited by 87
  9. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 85
  10. 2019Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewardscited by 78
  11. 2020Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcited by 60
  12. 2017Stein Variational Policy Gradientcited by 44
  13. 2021Annealed Stein Variational Gradient Descentcited by 26
  14. 2021Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problemscited by 26
  15. 2021Density-based Curriculum for Multi-goal Reinforcement Learning with Sparse Rewardscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2024S2AC: Energy-Based Reinforcement Learning with Stein Soft Actor Criticcited by 18
  2. 2025MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spacescited by 7
  3. 2026Improving Zero-Shot Offline RL via Behavioral Task Samplingcited by 1
  4. 2025Proposing Hierarchical Goal-Conditioned Policy Planning in Multi-Goal Reinforcement Learningcited by 0
  5. 2025Imagine Beyond! Distributionally Robust Auto-Encoding for State Space Coverage in Online Reinforcement Learningcited by 0
  6. 2026Self-Curriculum Model-based Reinforcement Learning for Shape Control of Deformable Linear Objectscited by 0