mlsanityData to 7 Oct 2026

What Makes a Good Diffusion Planner for Decision Making?

arXiv 2503.00535 ↗First posted 1 Mar 2025cs.LG cs.AIcited by 37

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Diffusion models have recently shown significant potential in solving decision-making problems, particularly in generating behavior plans -- also known as diffusion planning. While numerous studies have demonstrated the impressive performance of diffusion planning, the mechanisms behind the key components of a good diffusion planner remain unclear and the design choices are highly inconsistent in existing studies. In this work, we address this issue through systematic empirical experiments on diffusion planning in an offline reinforcement learning (RL) setting, providing practical insights into the essential components of diffusion planning. We trained and evaluated over 6,000 diffusion models, identifying the critical components such as guided sampling, network architecture, action generation and planning strategy. We revealed that some design choices opposite to the common practice in previous work in diffusion planning actually lead to better performance, e.g., unconditional sampling with selection can be better than guided sampling and Transformer outperforms U-Net as denoising network. Based on these insights, we suggest a simple yet strong diffusion planning baseline that achieves state-of-the-art results on standard offline RL benchmarks.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 44 of its 57 references that are in the map. First posted on arXiv 1 Mar 2025.

Builds on

44 of 57 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Simple Hierarchical Planning with Diffusion5 Jan
  2. 2023Diffusion Models for Reinforcement Learning: A Survey2 Nov
  3. 2023AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Model3 Oct
  4. 2023SafeDiffuser: Safe Planning with Diffusion Probabilistic Models31 May
  5. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning25 Apr
  6. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies20 Apr
  7. 2023AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners3 Feb
  8. 2023Imitating Human Behaviour with Diffusion Models25 Jan
  9. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov
  10. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning12 Aug

Other references · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  10. 2023Consistency Modelscited by 1,778
  11. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  12. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
22 more
  1. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  2. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  3. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  4. 2022Diffusion Models in Vision: A Surveycited by 1,008
  5. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  6. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  7. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  8. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  9. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  10. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  11. 2023Learning Interactive Real-World Simulatorscited by 440
  12. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
  13. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  14. 2020Explainable Reinforcement Learning: A Surveycited by 139
  15. 2020Deployment-Efficient Reinforcement Learning via Model-Based Offline Optimizationcited by 120
  16. 2024Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Processcited by 107
  17. 2019PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaoscited by 56
  18. 2024DiffuserLite: Towards Real-time Diffusion Planningcited by 40
  19. 2024CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Makingcited by 38
  20. 2023Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertaintycited by 8
  21. 2023Habits and goals in synergy: a variational Bayesian framework for behaviorcited by 3
  22. 2023Toward Open-ended Embodied Tasks Solvingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 17; 2026: 20.

37 papers.

Sort:

By area:

  1. 2025Generative Trajectory Stitching through Diffusion Compositioncited by 38
  2. 2025Prior-Guided Diffusion Planning for Offline Reinforcement Learningcited by 13
  3. 2025Enhanced DACER Algorithm with High Diffusion Efficiencycited by 13
  4. 2025State-Covering Trajectory Stitching for Diffusion Plannerscited by 11
  5. 2025One-Step Flow Q-Learning: Addressing the Diffusion Policy Bottleneck in Offline Reinforcement Learningcited by 9
  6. 2025Multi-agent Coordination via Flow Matchingcited by 9
  7. 2025Local Manifold Approximation and Projection for Manifold-Aware Diffusion Planningcited by 8
  8. 2025Fast Monte Carlo Tree Diffusion: 100x Speedup via Parallel Sparse Planningcited by 6
  9. 2026Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusioncited by 6
  10. 2026GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Modelscited by 5
  11. 2025Tree-Guided Diffusion Plannercited by 4
  12. 2025Extremum Flow Matching for Offline Goal Conditioned Reinforcement Learningcited by 3
25 more
  1. 2026Implicit Maximum Likelihood Estimation for Real-time Generative Model Predictive Controlcited by 3
  2. 2025MaskedManipulator: Versatile Whole-Body Manipulationcited by 2
  3. 2025Consistency Trajectory Planning: High-Quality and Efficient Trajectory Optimization for Offline Model-Based Reinforcement Learningcited by 2
  4. 2025Compositional Monte Carlo Tree Diffusion for Extendable Planningcited by 2
  5. 2026Compositional Visual Planning via Inference-Time Diffusion Scalingcited by 2
  6. 2026Truncated Rectified Flow Policy for Reinforcement Learning with One-Step Samplingcited by 2
  7. 2026CoFlow: Coordinated Few-Step Flow for Offline Multi-Agent Decision Makingcited by 2
  8. 2026Refining Compositional Diffusion for Reliable Long-Horizon Planningcited by 2
  9. 2026HDFlow: Hierarchical Diffusion-Flow Planning for Long-horizon Taskscited by 2
  10. 2025Actor-Critic without Actorcited by 1
  11. 2026DADP: Domain Adaptive Diffusion Policycited by 1
  12. 2026Direct Soft-Policy Sampling via Langevin Dynamicscited by 1
  13. 2026Improving Diffusion Planners by Self-Supervised Action Gating with Energiescited by 1
  14. 2025What Do You Need for Compositional Generalization in Diffusion Planning?cited by 0
  15. 2025Fast and Stable Diffusion Planning through Variational Adaptive Weightingcited by 0
  16. 2025Mixed-Density Diffuser: Efficient Planning with Non-Uniform Temporal Resolutioncited by 0
  17. 2026GenPlanner: From Noise to Plans -- Emergent Reasoning in Flow Matching and Diffusion Modelscited by 0
  18. 2026State-Action Inpainting Diffuser for Continuous Control with Delaycited by 0
  19. 2026Muninn: Your Trajectory Diffusion Model But Fastercited by 0
  20. 2026Generative Auto-Bidding with Unified Modeling and Explorationcited by 0
  21. 2026Cross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learningcited by 0
  22. 2026Efficient and Uncertainty-Aware Diffusion Framework for Offline-to-Online Reinforcement Learningcited by 0
  23. 2026Fast and Highly Expressive Policy Learning for Offline Reinforcement Learning via Bootstrapped Flow Q-Learningcited by 0
  24. 2026RS-Diffuser: Risk-Sensitive Diffusion Planning with Distributional Value Guidancecited by 0
  25. 2026Shortcut Trajectory Planning for Efficient Offline Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion Models as Optimizers for Efficient Planning in Offline RL
  2. 2026Diffusion Modulation via Environment Mechanism Modeling for Planning
  3. 2025Habitizing Diffusion Planning for Efficient and Effective Decision Making
  4. 2025Accelerating Diffusion Planners in Offline RL via Reward-Aware Consistency Trajectory Distillation
  5. 2024Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learning
  6. 2025Diffusion Models for Reinforcement Learning: Foundations, Taxonomy, and Development
  7. 2024Diffusion Model for Planning: A Systematic Literature Review
  8. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning