mlsanityData to 14 Oct 2026

Planning as Descent: Goal-Conditioned Latent Trajectory Synthesis in Learned Energy Landscapes

arXiv 2512.17846 ↗First posted 19 Dec 2025cs.RO cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

We present Planning as Descent (PaD), a framework for offline goal-conditioned reinforcement learning that grounds trajectory synthesis in verification. Instead of learning a policy or explicit planner, PaD learns a goal-conditioned energy function over entire latent trajectories, assigning low energy to feasible, goal-consistent futures. Planning is realized as gradient-based refinement in this energy landscape, using identical computation during training and inference to reduce train-test mismatch common in decoupled modeling pipelines. PaD is trained via self-supervised hindsight goal relabeling, shaping the energy landscape around the planning dynamics. At inference, multiple trajectory candidates are refined under different temporal hypotheses, and low-energy plans balancing feasibility and efficiency are selected. We evaluate PaD on OGBench cube manipulation tasks. When trained on narrow expert demonstrations, PaD achieves state-of-the-art 95% success, strongly outperforming prior methods that peak at 68%. Remarkably, training on noisy, suboptimal data further improves success and plan efficiency, highlighting the benefits of verification-driven planning. Our results suggest learning to evaluate and refine trajectories provides a robust alternative to direct policy learning for offline, reward-free planning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 33 of its 38 references that are in the map. First posted on arXiv 19 Dec 2025.

Builds on

33 of 38 references.

References · 33

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  7. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  8. 2017Hindsight Experience Replaycited by 1,136
  9. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  10. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
  11. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  12. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
21 more
  1. 2021Implicit Behavioral Cloningcited by 452
  2. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
  3. 2019Learning Latent Plans from Playcited by 316
  4. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  5. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  6. 2025Flow Q-Learningcited by 184
  7. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  8. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 125
  9. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  10. 2023The Generative AI Paradox: "What It Can Create, It May Not Understand"cited by 98
  11. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 89
  12. 2022A Survey on Model-based Reinforcement Learningcited by 66
  13. 2019Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Trafficcited by 50
  14. 2016Self-Correcting Models for Model-Based Reinforcement Learningcited by 43
  15. 2023Masked Trajectory Models for Prediction, Representation, and Controlcited by 42
  16. 2020Energy-Based Imitation Learningcited by 40
  17. 2025Energy-Based Transformers are Scalable Learners and Thinkerscited by 31
  18. 2022UniMASK: Unified Inference in Sequential Decision Problemscited by 24
  19. 2023Energy-Inspired Self-Supervised Pretraining for Vision Modelscited by 7
  20. 2025Model-Based Diffusion Sampling for Predictive Control in Offline Decision Makingcited by 3
  21. 2025EBT-Policy: Energy Unlocks Emergent Physical Reasoning Capabilitiescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving Diffusion Planners by Self-Supervised Action Gating with Energies
  2. 2026Reasoning as Energy Minimization over Structured Latent Trajectories
  3. 2026Latent Energy Action Planning with World Models
  4. 2025Test-Time Graph Search for Goal-Conditioned Reinforcement Learning
  5. 2024Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inference
  6. 2026Shortcut Trajectory Planning for Efficient Offline Reinforcement Learning
  7. 2026Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning
  8. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learning