mlsanityData to 7 Oct 2026

Backward Learning for Goal-Conditioned Policies

arXiv 2312.05044 ↗First posted 8 Dec 2023cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Can we learn policies in reinforcement learning without rewards? Can we learn a policy just by trying to reach a goal state? We answer these questions positively by proposing a multi-step procedure that first learns a world model that goes backward in time, secondly generates goal-reaching backward trajectories, thirdly improves those sequences using shortest path finding algorithms, and finally trains a neural network policy by imitation learning. We evaluate our method on a deterministic maze environment where the observations are 64× 64 pixel bird's eye images and can show that it consistently reaches several goals.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 25 of its 34 references that are in the map. First posted on arXiv 8 Dec 2023.

Builds on

25 of 34 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016Layer Normalizationcited by 5,579
  4. 2018World Modelscited by 1,444
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  6. 2023Mastering Diverse Domains through World Modelscited by 1,193
  7. 2017Hindsight Experience Replaycited by 1,096
  8. 2020Mastering Atari with Discrete World Modelscited by 990
  9. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  10. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  11. 2019Model-Based Reinforcement Learning for Ataricited by 495
  12. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
13 more
  1. 2020First return, then explorecited by 256
  2. 2023Transformer-based World Models Are Happy With 100k Interactionscited by 137
  3. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  4. 2019Imitation Learning from Imperfect Demonstrationcited by 112
  5. 2022Deep Hierarchical Planning from Pixelscited by 96
  6. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learningcited by 48
  7. 2021Offline Reinforcement Learning with Reverse Model-based Imaginationcited by 47
  8. 2020Bidirectional Model-based Policy Optimizationcited by 38
  9. 2018Recall Traces: Backtracking Models for Efficient Reinforcement Learningcited by 23
  10. 2018Forward-Backward Reinforcement Learningcited by 17
  11. 2020Forethought and Hindsight in Credit Assignmentcited by 16
  12. 2022Backward Imitation and Forward Reinforcement Learning via Bi-directional Model Rolloutscited by 2
  13. 2023Time-Myopic Go-Explore: Learning A State Representation for the Go-Explore Paradigmcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2024Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNetscited by 8
  2. 2026World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Learning to Reach Goals via Iterated Supervised Learning
  2. 2019Reward-Conditioned Policies
  3. 2019Goal-conditioned Imitation Learning
  4. 2019Learning from Trajectories via Subgoal Discovery
  5. 2023Learning to Reach Goals via Diffusion
  6. 2026Learning from the Near Future: Temporal Self-Distillation for RLVR
  7. 2018Fast Policy Learning through Imitation and Reinforcement
  8. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following