mlsanityData to 7 Oct 2026

Exploration via Hindsight Goal Generation

arXiv 1906.04279 ↗First posted 10 Jun 2019cs.LG stat.MLcited by 53

Not on the map. Why?

Abstract (arXiv)

Goal-oriented reinforcement learning has recently been a practical framework for robotic manipulation tasks, in which an agent is required to reach a certain goal defined by a function on the state space. However, the sparsity of such reward definition makes traditional reinforcement learning algorithms very inefficient. Hindsight Experience Replay (HER), a recent advance, has greatly improved sample efficiency and practical applicability for such problems. It exploits previous replays by constructing imaginary goals in a simple heuristic way, acting like an implicit curriculum to alleviate the challenge of sparse reward signal. In this paper, we introduce Hindsight Goal Generation (HGG), a novel algorithmic framework that generates valuable hindsight goals which are easy for an agent to achieve in the short term and are also potential for guiding the agent to reach the actual goal in the long term. We have extensively evaluated our goal generation algorithm on a number of robotic manipulation tasks and demonstrated substantially improvement over the original HER in terms of sample efficiency.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 8; 2023: 13; 2024: 3; 2025: 9; 2026: 8.

53 papers.

Sort:

By area:

  1. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  2. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  3. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  4. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 70
  5. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  6. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
  7. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  8. 2021Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problemscited by 25
  9. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learningcited by 22
  10. 2021Solving Compositional Reinforcement Learning Problems via Task Reductioncited by 19
  11. 2022Provably Safe Deep Reinforcement Learning for Robotic Manipulation in Human Environmentscited by 16
  12. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 16
38 more
  1. 2021C-Planning: An Automatic Curriculum for Learning Goal-Reaching Taskscited by 15
  2. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 14
  3. 2021MapGo: Model-Assisted Policy Optimization for Goal-Oriented Taskscited by 13
  4. 2021Learning Domain Invariant Representations in Goal-conditioned Block MDPscited by 12
  5. 2023CQM: Curriculum Reinforcement Learning with a Quantized World Modelcited by 11
  6. 2025DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learningcited by 11
  7. 2021Adjacency constraint for efficient hierarchical reinforcement learningcited by 10
  8. 2022Generative Planning for Temporally Coordinated Exploration in Reinforcement Learningcited by 10
  9. 2021DisCo RL: Distribution-Conditioned Reinforcement Learning for General-Purpose Policiescited by 9
  10. 2022State-Conditioned Adversarial Subgoal Generationcited by 8
  11. 2023Variational Curriculum Reinforcement Learning for Unsupervised Discovery of Skillscited by 8
  12. 2024RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanationcited by 7
  13. 2023On the Benefit of Optimal Transport for Curriculum Reinforcement Learningcited by 6
  14. 2023Building Open-Ended Embodied Agent via Language-Policy Bidirectional Adaptationcited by 6
  15. 2024Learning World Models for Unconstrained Goal Navigationcited by 6
  16. 2023Granger Causal Interaction Skill Chainscited by 5
  17. 2025Hierarchical Reinforcement Learning with Uncertainty-Guided Diffusional Subgoalscited by 5
  18. 2023Diversify & Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreementcited by 4
  19. 2026SVL: Goal-Conditioned Reinforcement Learning as Survival Learningcited by 3
  20. 2021Learning for Visual Navigation by Imagining the Successcited by 2
  21. 2021Long-Term Exploration in Persistent MDPscited by 2
  22. 2023CRISP: Curriculum Inducing Primitive Informed Subgoal Prediction for Hierarchical Reinforcement Learningcited by 2
  23. 2023Demonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculumcited by 2
  24. 2023Machine Learning Meets Advanced Robotic Manipulationcited by 2
  25. 2025Test-Time Alignment for Large Language Models via Textual Model Predictive Controlcited by 2
  26. 2025Dual-Objective Reinforcement Learning with Novel Hamilton-Jacobi-Bellman Formulationscited by 2
  27. 2025Learning Instruction-Following Policies through Open-Ended Instruction Relabeling with Large Language Modelscited by 2
  28. 2025Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machinescited by 2
  29. 2023Hindsight States: Blending Sim and Real Task Elements for Efficient Reinforcement Learningcited by 1
  30. 2023Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularizationcited by 1
  31. 2024Exploring the Edges of Latent State Clusters for Goal-Conditioned Reinforcement Learningcited by 1
  32. 2025Strict Subgoal Execution: Reliable Long-Horizon Planning in Hierarchical Reinforcement Learningcited by 1
  33. 2026Action-Sufficient Goal Representationscited by 1
  34. 2026Improving Zero-Shot Offline RL via Behavioral Task Samplingcited by 1
  35. 2023Learning and reusing primitive behaviours to improve Hindsight Experience Replay sample efficiencycited by 0
  36. 2025Imagine Beyond! Distributionally Robust Auto-Encoding for State Space Coverage in Online Reinforcement Learningcited by 0
  37. 2026Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Samplingcited by 0
  38. 2026Rotation-Aware Point-Cloud Embeddings for Vision-Based In-Hand Reorientationcited by 0

50 most cited of 53 All 53 (JSON)