mlsanityData to 14 Oct 2026

Policy Continuation with Hindsight Inverse Dynamics

arXiv 1910.14055 ↗First posted 30 Oct 2019cs.LG stat.MLcited by 22

Not on the map. Why?

Abstract (arXiv)

Solving goal-oriented tasks is an important but challenging problem in reinforcement learning (RL). For such tasks, the rewards are often sparse, making it difficult to learn a policy effectively. To tackle this difficulty, we propose a new approach called Policy Continuation with Hindsight Inverse Dynamics (PCHID). This approach learns from Hindsight Inverse Dynamics based on Hindsight Experience Replay, enabling the learning process in a self-imitated manner and thus can be trained with supervised learning. This work also extends it to multi-step settings with Policy Continuation. The proposed method is general, which can work in isolation or be combined with other on-policy and off-policy algorithms. On two multi-goal tasks GridWorld and FetchReach, PCHID significantly improves the sample efficiency as well as the final performance.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 7; 2023: 3; 2024: 5; 2025: 2; 2026: 1.

22 papers.

Sort:

By area:

  1. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  2. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 175
  3. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  4. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 121
  5. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  6. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  7. 2022How to Leverage Unlabeled Data in Offline Reinforcement Learningcited by 62
  8. 2023Contrastive Difference Predictive Codingcited by 36
  9. 2022Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Spacecited by 31
  10. 2024Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of Viewcited by 27
  11. 2022Generalization with Lossy Affordances: Leveraging Broad Offline Data for Learning Visuomotor Taskscited by 25
  12. 2022Imitating Past Successes can be Very Suboptimalcited by 23
10 more
  1. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 23
  2. 2023Reinforcement Learning in the Era of LLMs: What is Essential? What is needed? An RL Perspective on RLHF, Prompting, and Beyondcited by 20
  3. 2021MHER: Model-based Hindsight Experience Replaycited by 15
  4. 2024Solving Minimum-Cost Reach Avoid using Reinforcement Learningcited by 14
  5. 2023Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examplescited by 9
  6. 2022Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shapingcited by 6
  7. 2025Efficient Skill Discovery via Regret-Aware Optimizationcited by 3
  8. 2025Self-Supervised Goal-Reaching Results in Multi-Agent Cooperation and Explorationcited by 3
  9. 2024Urban-Focused Multi-Task Offline Reinforcement Learning with Contrastive Data Sharingcited by 2
  10. 2026Self-Supervised On-Policy Reinforcement Learning via Contrastive Proximal Policy Optimisationcited by 0