mlsanityData to 7 Oct 2026

Efficient Planning with Latent Diffusion

arXiv 2310.00311 ↗First posted 30 Sep 2023cs.LG cs.AIcited by 11

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Temporal abstraction and efficient planning pose significant challenges in offline reinforcement learning, mainly when dealing with domains that involve temporally extended tasks and delayed sparse rewards. Existing methods typically plan in the raw action space and can be inefficient and inflexible. Latent action spaces offer a more flexible paradigm, capturing only possible actions within the behavior policy support and decoupling the temporal structure between planning and modeling. However, current latent-action-based methods are limited to discrete spaces and require expensive planning. This paper presents a unified framework for continuous latent action space representation learning and planning by leveraging latent, score-based diffusion models. We establish the theoretical equivalence between planning in the latent action space and energy-guided sampling with a pretrained diffusion model and incorporate a novel sequence-level exact sampling method. Our proposed method, textttLatentDiffuser, demonstrates competitive performance on low-dimensional locomotion control tasks and surpasses existing methods in higher-dimensional tasks.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 104 of its 127 references that are in the map. First posted on arXiv 30 Sep 2023.

Builds on

104 of 127 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning10 Jul
  2. 2023Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning8 Jun
  3. 2023MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RL31 May
  4. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning29 May
  5. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning25 Apr
  6. 2023AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners3 Feb
  7. 2023Imitating Human Behaviour with Diffusion Models25 Jan
  8. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov
  9. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning12 Aug

Other references · 95

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2013Auto-Encoding Variational Bayescited by 3,980
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  12. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
38 more
  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2023Consistency Modelscited by 1,778
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2018Group Normalizationcited by 1,514
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  7. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  9. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  10. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  11. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  12. 2023Mastering Diverse Domains through World Modelscited by 1,193
  13. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  14. 2017Hindsight Experience Replaycited by 1,096
  15. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  16. 2020Mastering Atari with Discrete World Modelscited by 990
  17. 2022Denoising Diffusion Restoration Modelscited by 951
  18. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  19. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  20. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  21. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  22. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  23. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  24. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  25. 2021Score-based Generative Modeling in Latent Spacecited by 634
  26. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  27. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  28. 2016The Option-Critic Architecturecited by 433
  29. 2019Learning Latent Plans from Playcited by 310
  30. 2020Critic Regularized Regressioncited by 269
  31. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  32. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  33. 2022Diffusion models as plug-and-play priorscited by 230
  34. 2022Online Decision Transformercited by 194
  35. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  36. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
  37. 2022EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equationscited by 172
  38. 2018Deep Reinforcement Learning and the Deadly Triadcited by 170

50 most cited of 95 All 95 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 5; 2026: 4.

11 papers.

  1. 2024Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimizationcited by 99
  2. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 84
  3. 2025Revisiting Multi-Agent World Modeling from a Diffusion-Inspired Perspectivecited by 9
  4. 2025Local Manifold Approximation and Projection for Manifold-Aware Diffusion Planningcited by 8
  5. 2025DmC: Nearest Neighbor Guidance Diffusion Model for Offline Cross-domain Reinforcement Learningcited by 7
  6. 2026Structured 4D Latent Predictive Model for Robot Planningcited by 2
  7. 2025Diffusion Self-Weighted Guidance for Offline Reinforcement Learningcited by 1
  8. 2025Decision Flow Policy Optimizationcited by 0
  9. 2026Diffusion Modulation via Environment Mechanism Modeling for Planningcited by 0
  10. 2026Reasoning as Energy Minimization over Structured Latent Trajectoriescited by 0
  11. 2026Decoupled Guidance Diffusion for Adaptive Offline Safe Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Reasoning with Latent Diffusion in Offline Reinforcement Learning
  2. 2023Score Regularized Policy Optimization through Diffusion Behavior
  3. 2025Prior-Guided Diffusion Planning for Offline Reinforcement Learning
  4. 2022Leveraging Demonstrations with Latent Space Priors
  5. 2024Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inference
  6. 2026Improving Diffusion Planners by Self-Supervised Action Gating with Energies
  7. 2025State-Covering Trajectory Stitching for Diffusion Planners
  8. 2021Temporal Predictive Coding For Model-Based Planning In Latent Space