mlsanityData to 14 Oct 2026

Direction-Conditioned Policies for Online Goal-Conditioned Reinforcement Learning

arXiv 2610.05087 ↗First posted 4 Oct 2026cs.LG cs.AI cs.ROcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations also fits: RL for robot locomotion and manipulation

Abstract (arXiv)

Contrastive Reinforcement Learning (CRL) learns representations that estimate goal reachability, yet its policy remains conditioned on raw goals and therefore does not directly exploit the geometry encoded by its critic. We introduce Direction-Conditioned Policies (DCP), a method built around a small modification to CRL: DCP selects previously visited states as waypoints during online training and conditions the policy on their direction and distance in representation space. At deployment, DCP applies the same interface directly to the final goal, requiring neither waypoint selection nor planning. Across nine navigation and manipulation tasks, DCP attains higher final success rates than CRL on seven tasks and spends more time near the goal on seven. Controlled maze experiments further show that DCP captures shortest-path geometry more accurately and that the supplied direction causally influences the actor's behavior. We identify waypoint coverage and ranking as limits to exploration, and show that learned candidate generation improves goal reaching in two controlled mazes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 26 of its 27 references that are in the map. First posted on arXiv 4 Oct 2026.

Builds on

26 of 27 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Three Steps at a Time: Learning Representations from Action Sequences in Contrastive RL31 Aug
  2. 2025Dual Goal Representations8 Oct
  3. 2025Flattening Hierarchies with Policy Bootstrapping20 May
  4. 20251000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities19 Mar
  5. 2025Horizon Generalization in Reinforcement Learning6 Jan
  6. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making24 Jun
  7. 2024Foundation Policies with Hilbert Representations23 Feb

Other references · 19

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  3. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  4. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  5. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  6. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  7. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 194
  8. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  9. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  10. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  11. 2019Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discoverycited by 68
  12. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 62
7 more
  1. 2020Sub-Goal Trees -- a Framework for Goal-Based Reinforcement Learningcited by 29
  2. 2025Offline Goal-conditioned Reinforcement Learning with Quasimetric Representationscited by 24
  3. 2023Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Datacited by 23
  4. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 23
  5. 2024Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inferencecited by 17
  6. 2022Metric Residual Networks for Sample Efficient Goal-Conditioned Reinforcement Learningcited by 13
  7. 2019Plan Arithmetic: Compositional Plan Vectors for Multi-Task Controlcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Direction-Conditioned Policies via Compositional Subgoal Scoring for Online Goal-Conditioned Reinforcement Learning
  2. 2026Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?
  3. 2025Test-Time Graph Search for Goal-Conditioned Reinforcement Learning
  4. 2023Backward Learning for Goal-Conditioned Policies
  5. 2026Hyperspherical Contrastive Representations Unlock Planning and Exploration
  6. 2025Equivariant Goal Conditioned Contrastive Reinforcement Learning
  7. 2026Dual Advantage Fields
  8. 2022Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space