mlsanityData to 7 Oct 2026

C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks

arXiv 2110.12080 ↗First posted 22 Oct 2021cs.LG cs.AIcited by 15

Not on the map. Why?

Abstract (arXiv)

Goal-conditioned reinforcement learning (RL) can solve tasks in a wide range of domains, including navigation and manipulation, but learning to reach distant goals remains a central challenge to the field. Learning to reach such goals is particularly hard without any offline data, expert demonstrations, and reward shaping. In this paper, we propose an algorithm to solve the distant goal-reaching task by using search at training time to automatically generate a curriculum of intermediate states. Our algorithm, Classifier-Planning (C-Planning), frames the learning of the goal-conditioned policies as expectation maximization: the E-step corresponds to planning an optimal sequence of waypoints using graph search, while the M-step aims to learn a goal-conditioned policy to reach those waypoints. Unlike prior methods that combine goal-conditioned RL with graph search, ours performs search only during training and not testing, significantly decreasing the compute costs of deploying the learned policy. Empirically, we demonstrate that our method is more sample efficient than prior methods. Moreover, it is able to solve very long horizons manipulation and navigation tasks, tasks that prior goal-conditioned methods and methods based on graph search fail to solve.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 22 Oct 2021.

Builds on

29 of 41 references.

References · 29

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  4. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  5. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  6. 2019Learning Latent Plans from Playcited by 310
  7. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  8. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  9. 2018Semi-parametric Topological Memory for Navigationcited by 203
  10. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  11. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  12. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
17 more
  1. 2019Goal-conditioned Imitation Learningcited by 159
  2. 2019Planning with Goal-Conditioned Policiescited by 144
  3. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  4. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 100
  5. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 99
  6. 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 86
  7. 2020COG: Connecting New Skills to Past Experience with Offline Reinforcement Learningcited by 83
  8. 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 77
  9. 2020DisCor: Corrective Feedback in Reinforcement Learning via Distribution Correctioncited by 71
  10. 2019Hyperbolic Discounting and Learning over Multiple Horizonscited by 58
  11. 2019Exploration via Hindsight Goal Generationcited by 53
  12. 2019Mapping State Space using Landmarks for Universal Goal Reachingcited by 44
  13. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  14. 2019Scaling Local Control to Large-Scale Topological Navigationcited by 37
  15. 2020BeBold: Exploration Beyond the Boundary of Explored Regionscited by 32
  16. 2019Reinforcement Learning without Ground-Truth Statecited by 15
  17. 2020Universal Value Density Estimation for Imitation Learning and Goal-Conditioned Reinforcement Learningcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 4; 2025: 3; 2026: 2.

15 papers.

Sort:

By area:

  1. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  2. 2023Decomposing the Enigma: Subgoal-based Demonstration Learning for Formal Theorem Provingcited by 37
  3. 2022Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Spacecited by 31
  4. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 20
  5. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 16
  6. 2024Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inferencecited by 15
  7. 2025Horizon Generalization in Reinforcement Learningcited by 15
  8. 2025Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Followingcited by 13
  9. 2025Flattening Hierarchies with Policy Bootstrappingcited by 8
  10. 2023Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planningcited by 3
  11. 2023SEGO: Sequential Subgoal Optimization for Mathematical Problem-Solvingcited by 1
  12. 2024Single-Reset Divide & Conquer Imitation Learningcited by 0
3 more
  1. 2024Probabilistic World Modeling with Asymmetric Distance Measurecited by 0
  2. 2026Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Samplingcited by 0
  3. 2026Curriculum reinforcement learning with measurable task representation learningcited by 0