mlsanityData to 7 Oct 2026

Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning

arXiv 1811.09083 ↗First posted 22 Nov 2018cs.LG stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automatically learns a good representation of sub-goals in the environment and a low-level policy that can execute them. A high-level policy can then direct the lower one by generating a sequence of continuous sub-goal vectors. We evaluate our model using Mazebase and Mujoco environments, including the challenging AntGather task. Visualizations of the sub-goal embeddings reveal a logical decomposition of tasks within the environment. Quantitatively, our approach obtains compelling performance gains over non-hierarchical approaches.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 5; 2022: 7; 2023: 3; 2024: 4; 2025: 2; 2026: 1.

23 papers.

Sort:

By area:

  1. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  2. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 46
  3. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  4. 2021Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraftcited by 34
  5. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  6. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 16
  7. 2021Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learningcited by 15
  8. 2025RL for Reasoning by Adaptively Revealing Rationalescited by 15
  9. 2022It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generationcited by 14
  10. 2024Learning to Drive via Asymmetric Self-Playcited by 11
  11. 2022Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learningcited by 10
  12. 2024Probabilistic Subgoal Representations for Hierarchical Reinforcement learningcited by 8
11 more
  1. 2020Augmenting GAIL with BC for sample efficient imitation learningcited by 4
  2. 2023Towards Skilled Population Curriculum for Multi-Agent Reinforcement Learningcited by 4
  3. 2022Walk the Random Walk: Learning to Discover and Reach Goals Without Supervisioncited by 3
  4. 2022Active Task Randomization: Learning Robust Skills via Unsupervised Generation of Diverse and Feasible Taskscited by 3
  5. 2024Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generationcited by 3
  6. 2021Eden: A Unified Environment Framework for Booming Reinforcement Learning Algorithmscited by 2
  7. 2023A framework for the emergence and analysis of language in social learning agentscited by 2
  8. 2024Scalable Interactive Machine Learning for Future Command and Controlcited by 2
  9. 2025CORD: Generalizable Cooperation via Role Diversitycited by 1
  10. 2022AutoDIME: Automatic Design of Interesting Multi-Agent Environmentscited by 0
  11. 2026Unsupervised Learning of Efficient Exploration: Pre-training Adaptive Policies via Self-Imposed Goalscited by 0