mlsanityData to 7 Oct 2026

AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners

arXiv 2302.01877 ↗First posted 3 Feb 2023cs.LGcited by 147

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Diffusion models have demonstrated their powerful generative capability in many tasks, with great potential to serve as a paradigm for offline reinforcement learning. However, the quality of the diffusion model is limited by the insufficient diversity of training data, which hinders the performance of planning and the generalizability to new tasks. This paper introduces AdaptDiffuser, an evolutionary planning method with diffusion that can self-evolve to improve the diffusion model hence a better planner, not only for seen tasks but can also adapt to unseen tasks. AdaptDiffuser enables the generation of rich synthetic expert data for goal-conditioned tasks using guidance from reward gradients. It then selects high-quality data via a discriminator to finetune the diffusion model, which improves the generalization ability to unseen tasks. Empirical experiments on two benchmark environments and two carefully designed unseen tasks in KUKA industrial robot arm and Maze2D environments demonstrate the effectiveness of AdaptDiffuser. For example, AdaptDiffuser not only outperforms the previous art Diffuser by 20.8% on Maze2D and 7.5% on MuJoCo locomotion, but also adapts better to new tasks, e.g., KUKA pick-and-place, by 27.9% without requiring additional expert data. More visualization results and demo videos could be found on our project page.

Key facts. Cited by 147 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 34 of its 69 references that are in the map. First posted on arXiv 3 Feb 2023.

Builds on

34 of 69 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov
  2. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning12 Aug

Other references · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  11. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  12. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
20 more
  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  3. 2018Group Normalizationcited by 1,514
  4. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  6. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  7. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  8. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  9. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  10. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  11. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  12. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  13. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  14. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
  15. 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 147
  16. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 147
  17. 2020Model-Based Offline Planningcited by 126
  18. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
  19. 2020The Importance of Pessimism in Fixed-Dataset Policy Optimizationcited by 99
  20. 2022Bootstrapped Transformer for Offline Reinforcement Learningcited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 52; 2025: 54; 2026: 25.

147 papers.

Sort:

By area:

  1. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  2. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  3. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  4. 2024Diffusion Policy Policy Optimizationcited by 255
  5. 2024RoboDreamer: Learning Compositional World Models for Robot Imaginationcited by 187
  6. 2025Flow Q-Learningcited by 162
  7. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  8. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
  9. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  10. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  11. 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
  12. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 84
38 more
  1. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  2. 2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 69
  3. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 68
  4. 2023Elastic Decision Transformercited by 66
  5. 2024Equivariant Diffusion Policycited by 61
  6. 2023Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimizationcited by 56
  7. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learningcited by 56
  8. 2023Reward-Directed Conditional Diffusion: Provable Distribution Estimation and Reward Improvementcited by 49
  9. 2023EDMP: Ensemble-of-costs-guided Diffusion for Motion Planningcited by 49
  10. 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Modelscited by 47
  11. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  12. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
  13. 2024Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Followingcited by 41
  14. 2024DiffuserLite: Towards Real-time Diffusion Planningcited by 40
  15. 2024Potential Based Diffusion Motion Planningcited by 40
  16. 2025Diffusion Models for Robotic Manipulation: A Surveycited by 39
  17. 2024CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Makingcited by 38
  18. 2025Generative Trajectory Stitching through Diffusion Compositioncited by 38
  19. 2023AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Modelcited by 37
  20. 2025What Makes a Good Diffusion Planner for Decision Making?cited by 37
  21. 2024DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitchingcited by 35
  22. 2024G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulationcited by 35
  23. 2024Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learningcited by 34
  24. 2023Adaptive Online Replanning with Diffusion Modelscited by 33
  25. 2024Understanding and Improving Training-free Loss-based Diffusion Guidancecited by 31
  26. 2024Model-Based Diffusion for Trajectory Optimizationcited by 30
  27. 2023DiffCPS: Diffusion Model based Constrained Policy Search for Offline Reinforcement Learningcited by 27
  28. 2023Refining Diffusion Planner for Reliable Behavior Synthesis by Automatic Detection of Infeasible Planscited by 27
  29. 2024Diffusion Model for Data-Driven Black-Box Optimizationcited by 24
  30. 2024DNAct: Diffusion Guided Multi-Task 3D Policy Learningcited by 22
  31. 2024GTA: Generative Trajectory Augmentation with Guidance for Offline Reinforcement Learningcited by 22
  32. 2024Stitching Sub-Trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RLcited by 20
  33. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  34. 2024DiffuSolve: Diffusion-based Solver for Non-convex Trajectory Optimizationcited by 19
  35. 2024AlignIQL: Policy Alignment in Implicit Q-Learning through Constrained Optimizationcited by 19
  36. 2024Generative Image as Action Modelscited by 18
  37. 2023Learning to Read Braille: Bridging the Tactile Reality Gap with Diffusion Modelscited by 17
  38. 2023Extracting Reward Functions from Diffusion Modelscited by 17

50 most cited of 147 All 147 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RL
  2. 2024Simple Hierarchical Planning with Diffusion
  3. 2025Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making
  4. 2025DiWA: Diffusion Policy Adaptation with World Models
  5. 2026Guided Discovery of New Behaviors using Diffusion Policies
  6. 2026Beyond Imitation: Reinforcement Learning Fine-Tuning for Adaptive Diffusion Navigation Policies
  7. 2023SafeDiffuser: Safe Planning with Diffusion Probabilistic Models
  8. 2025Exploration and Adaptation in Non-Stationary Tasks with Diffusion Policies