mlsanityData to 7 Oct 2026

Hierarchical Reinforcement Learning with Hindsight

arXiv 1805.08180 ↗First posted 21 May 2018cs.LG cs.AI cs.NEcited by 26

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning (RL) algorithms can suffer from poor sample efficiency when rewards are delayed and sparse. We introduce a solution that enables agents to learn temporally extended actions at multiple levels of abstraction in a sample efficient and automated fashion. Our approach combines universal value functions and hindsight learning, allowing agents to learn policies belonging to different time scales in parallel. We show that our method significantly accelerates learning in a variety of discrete and continuous tasks.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 4; 2023: 3; 2024: 2; 2025: 5; 2026: 6.

26 papers.

Sort:

By area:

  1. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  2. 2021Successor Feature Landmarks for Long-Horizon Goal-Conditioned Reinforcement Learningcited by 23
  3. 2025Efficient Robotic Policy Learning via Latent Space Backward Planningcited by 14
  4. 2021Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Taskscited by 12
  5. 2024ALaRM: Align Language Models via Hierarchical Rewards Modelingcited by 11
  6. 2024SkiLD: Unsupervised Skill Discovery Guided by Factor Interactionscited by 9
  7. 2021Context-Specific Representation Abstraction for Deep Option Learningcited by 8
  8. 2021DisTop: Discovering a Topological representation to learn diverse and rewarding skillscited by 7
  9. 2023Story Shaping: Teaching Agents Human-like Behavior with Storiescited by 7
  10. 2021Toward Robust Long Range Policy Transfercited by 6
  11. 2025Safe Multi-Agent Navigation guided by Goal-Conditioned Safe Reinforcement Learningcited by 5
  12. 2022Human-AI Shared Control via Policy Dissectioncited by 4
14 more
  1. 2026Laplacian Representations for Decision-Time Planningcited by 4
  2. 2026Compositional Planning with Jumpy World Modelscited by 4
  3. 2021HAC Explore: Accelerating Exploration with Hierarchical Reinforcement Learningcited by 3
  4. 2023Efficient Learning of High Level Plans from Playcited by 2
  5. 2025Generalizable Hierarchical Skill Learning via Object-Centric Representationcited by 2
  6. 2022Reinforcement Learning with Intrinsic Affinity for Personalized Prosperity Managementcited by 1
  7. 2022Syntactic Inductive Biases for Deep Learning Methodscited by 0
  8. 2023ETHER: Aligning Emergent Communication for Hindsight Experience Replaycited by 0
  9. 2025A Unified Framework for Zero-Shot Reinforcement Learningcited by 0
  10. 2025SCOPE: Language Models as One-Time Teacher for Hierarchical Planning in Text Environmentscited by 0
  11. 2026Exploiting Local Dynamics Regularity for Reusable Skills in Offline Hierarchical RLcited by 0
  12. 2026Imagine to Ensure Safety in Hierarchical Reinforcement Learningcited by 0
  13. 2026Path-level Hindsight Instructions for Semantic Exploration in Vision-Language Navigationcited by 0
  14. 2026Improving Offline Goal-Conditioned Reinforcement Learning via Selective Reward Stimulationcited by 0