mlsanityData to 14 Oct 2026

Planning with Diffusion for Flexible Behavior Synthesis

arXiv 2205.09991 ↗First posted 20 May 2022cs.LG cs.AIcited by 1,061

Not on the map. Why?

Cited by 565 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Model-based reinforcement learning methods often use learning only for the purpose of estimating an approximate dynamics model, offloading the rest of the decision-making work to classical trajectory optimizers. While conceptually simple, this combination has a number of empirical shortcomings, suggesting that learned models may not be well-suited to standard trajectory optimization. In this paper, we consider what it would look like to fold as much of the trajectory optimization pipeline as possible into the modeling problem, such that sampling from the model and planning with it become nearly identical. The core of our technical approach lies in a diffusion probabilistic model that plans by iteratively denoising trajectories. We show how classifier-guided sampling and image inpainting can be reinterpreted as coherent planning strategies, explore the unusual and useful properties of diffusion-based planning methods, and demonstrate the effectiveness of our framework in control settings that emphasize long-horizon decision-making and test-time flexibility.

Key facts. Cited by 1,061 papers in the map (data through 14 Oct 2026). Builds on 50 of its 76 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

50 of 76 references.

References · 50

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  9. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  11. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  12. 2018Group Normalizationcited by 1,524
38 more
  1. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  3. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  6. 2017Hindsight Experience Replaycited by 1,136
  7. 2020Mastering Atari with Discrete World Modelscited by 1,007
  8. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  9. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  10. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  11. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  12. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  13. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  14. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  15. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  16. 2019Model-Based Reinforcement Learning for Ataricited by 502
  17. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  18. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  19. 2018Graph networks as learnable physics engines for inference and controlcited by 244
  20. 2019Benchmarking Model-Based Reinforcement Learningcited by 162
  21. 2016Value Iteration Networkscited by 160
  22. 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 150
  23. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 148
  24. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  25. 2019Mish: A Self Regularized Non-Monotonic Activation Functioncited by 141
  26. 2020Model-Based Offline Planningcited by 129
  27. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 124
  28. 2017Value Prediction Networkcited by 117
  29. 20213D Neural Scene Representations for Visuomotor Controlcited by 111
  30. 2018Deep Imitative Models for Flexible Inference, Planning, and Controlcited by 76
  31. 2019Structured agents for physical constructioncited by 50
  32. 2021Vector Quantized Models for Planningcited by 48
  33. 2020Generative Temporal Difference Learning for Infinite-Horizon Predictioncited by 45
  34. 2021Mismatched No More: Joint Model-Policy Optimization for Model-Based RLcited by 36
  35. 2019Model Based Planning with Energy Based Modelscited by 33
  36. 2020Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEscited by 32
  37. 2021Model-Based Reinforcement Learning via Latent-Space Collocationcited by 28
  38. 2020Learning Accurate Long-term Dynamics for Model-based Reinforcement Learningcited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 21; 2023: 135; 2024: 246; 2025: 346; 2026: 313.

1,061 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  3. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  4. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  5. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  6. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  7. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  8. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  9. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  10. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
  11. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 325
  12. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
38 more
  1. 2022EDGE: Editable Dance Generation From Musiccited by 287
  2. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 283
  3. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 282
  4. 2024Diffusion Policy Policy Optimizationcited by 280
  5. 2023Imitating Human Behaviour with Diffusion Modelscited by 263
  6. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 252
  7. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 248
  8. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  9. 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policiescited by 236
  10. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 230
  11. 2023Guided Motion Diffusion for Controllable Human Motion Synthesiscited by 223
  12. 2025Real-Time Execution of Action Chunking Flow Policiescited by 216
  13. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 213
  14. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 190
  15. 2025Flow Q-Learningcited by 184
  16. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 182
  17. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 181
  18. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 177
  19. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 169
  20. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 165
  21. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  22. 2023Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Modelscited by 161
  23. 2023AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Plannerscited by 149
  24. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 149
  25. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 146
  26. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  27. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 141
  28. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 138
  29. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
  30. 2023Learning a Diffusion Model Policy from Rewards via Q-Score Matchingcited by 138
  31. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  32. 2023Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Modelscited by 133
  33. 2022SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusioncited by 127
  34. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  35. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 121
  36. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 116
  37. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 115
  38. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 112

50 most cited of 1,061 All 1,061 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Imitation: Reinforcement Learning Fine-Tuning for Adaptive Diffusion Navigation Policies
  2. 2026Guided Discovery of New Behaviors using Diffusion Policies
  3. 2026Model-Based Diffusion Optimal Control for Multi-Robot Motion Planning
  4. 2026Diffusion Policy through Conditional Proximal Policy Optimization
  5. 2026Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View
  6. 2024Task-Agnostic Pre-training and Task-Guided Fine-tuning for Versatile Diffusion Planner
  7. 2025CCDP: Composition of Conditional Diffusion Policies with Guided Sampling
  8. 2026Diffusion Controller: Framework, Algorithms and Parameterization