mlsanityData to 7 Oct 2026

Generalized Hindsight for Reinforcement Learning

arXiv 2002.11708 ↗First posted 26 Feb 2020cs.LG cs.AI cs.NEcited by 51

Not on the map. Why?

Abstract (arXiv)

One of the key reasons for the high sample complexity in reinforcement learning (RL) is the inability to transfer knowledge from one task to another. In standard multi-task RL settings, low-reward data collected while trying to solve one task provides little to no signal for solving that particular task and is hence effectively wasted. However, we argue that this data, which is uninformative for one task, is likely a rich source of information for other tasks. To leverage this insight and efficiently reuse data, we present Generalized Hindsight: an approximate inverse reinforcement learning technique for relabeling behaviors with the right tasks. Intuitively, given a behavior generated under one task, Generalized Hindsight returns a different task that the behavior is better suited for. Then, the behavior is relabeled with this new task before being used by an off-policy RL optimizer. Compared to standard relabeling techniques, Generalized Hindsight provides a substantially more efficient reuse of samples, which we empirically demonstrate on a suite of multi-task navigation and manipulation tasks. Videos and code can be accessed here: https://sites.google.com/view/generalized-hindsight.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 14; 2023: 11; 2024: 9; 2025: 5; 2026: 3.

51 papers.

Sort:

By area:

  1. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  2. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 171
  3. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  4. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  5. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learningcited by 93
  6. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  7. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learningcited by 64
  8. 2022How to Leverage Unlabeled Data in Offline Reinforcement Learningcited by 61
  9. 2023What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?cited by 36
  10. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  11. 2024Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodingscited by 31
  12. 2023Cross-Domain Policy Adaptation via Value-Guided Data Filteringcited by 30
38 more
  1. 2021StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learningcited by 27
  2. 2022The Challenges of Exploration for Offline Reinforcement Learningcited by 26
  3. 2022Imitating Past Successes can be Very Suboptimalcited by 20
  4. 2023Graph Decision Transformercited by 20
  5. 2023Distance Weighted Supervised Learning for Offline Interaction Datacited by 18
  6. 2024Reinforcing Language Agents via Policy Optimization with Action Decompositioncited by 16
  7. 2021MHER: Model-based Hindsight Experience Replaycited by 14
  8. 2022It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generationcited by 14
  9. 2024Offline Multitask Representation Learning for Reinforcement Learningcited by 14
  10. 2021Wish you were here: Hindsight Goal Selection for long-horizon dexterous manipulationcited by 13
  11. 2022A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcited by 12
  12. 2023Visual Affordance Prediction for Guiding Robot Explorationcited by 12
  13. 2024Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcementcited by 12
  14. 2022One After Another: Learning Incremental Skills for a Changing Worldcited by 10
  15. 2023The Provable Benefits of Unsupervised Data Sharing for Offline Reinforcement Learningcited by 10
  16. 2021Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RLcited by 8
  17. 2021Hindsight Goal Ranking on Replay Buffer for Sparse Reward Environmentcited by 7
  18. 2023Understanding when Dynamics-Invariant Data Augmentations Benefit Model-Free Reinforcement Learning Updatescited by 6
  19. 2024Pessimistic Value Iteration for Multi-Task Data Sharing in Offline Reinforcement Learningcited by 6
  20. 2022Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Taskscited by 5
  21. 2025WebDART: Dynamic Decomposition and Re-planning for Complex Web Taskscited by 5
  22. 2021Hindsight Foresight Relabeling for Meta-Reinforcement Learningcited by 4
  23. 2021Learning from demonstrations with SACR2: Soft Actor-Critic with Reward Relabelingcited by 4
  24. 2023DREAM: Adaptive Reinforcement Learning based on Attention Mechanism for Temporal Knowledge Graph Reasoningcited by 4
  25. 2024Improving Sample Efficiency of Reinforcement Learning with Background Knowledge from Large Language Modelscited by 4
  26. 2025Null Counterfactual Factor Interactions for Goal-Conditioned Reinforcement Learningcited by 4
  27. 2023Machine Learning Meets Advanced Robotic Manipulationcited by 2
  28. 2025GCHR : Goal-Conditioned Hindsight Regularization for Sample-Efficient Reinforcement Learningcited by 2
  29. 2022STIR2: Reward Relabelling for combined Reinforcement and Imitation Learning on sparse-reward taskscited by 1
  30. 2022Reinforcement Learning Textbookcited by 1
  31. 2022Executive Function: A Contrastive Value Policy for Resampling and Relabeling Perceptions via Hindsight Summarization?cited by 1
  32. 2023Do as I can, not as I getcited by 1
  33. 2024Provable Interactive Learning with Hindsight Instruction Feedbackcited by 1
  34. 2024How to Solve Contextual Goal-Oriented Problems with Offline Datasets?cited by 1
  35. 2025DAIL: Beyond Task Ambiguity for Language-Conditioned Reinforcement Learningcited by 1
  36. 2025NFQ2.0: The CartPole Benchmark Revisitedcited by 1
  37. 2026Can We Really Learn One Representation to Optimize All Rewards?cited by 1
  38. 2026STO-RL: Offline RL under Sparse Rewards via LLM-Guided Subgoal Temporal Ordercited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Understanding Hindsight Goal Relabeling from a Divergence Minimization Perspective
  2. 2026Learning More from Less: Reinforcement Learning from Hindsight
  3. 2026Where the Model Changes Its Mind: Hindsight-Divergence Localization for Efficient Reinforcement Learning with Verifiable Rewards
  4. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review
  5. 2026HindSearch: Trajectory-Level Hindsight Critique for Search-Augmented Reinforcement Learning
  6. 2026Trajectory-Relative Hindsight Distillation for Agentic Reinforcement Learning
  7. 2020Generalized Inverse Planning: Learning Lifted non-Markovian Utility for Generalizable Task Representation ↗
  8. 2025Replay Failures as Successes: Sample-Efficient Reinforcement Learning for Instruction Following