mlsanityData to 7 Oct 2026

MetaDiffuser: Diffusion Model as Conditional Planner for Offline Meta-RL

arXiv 2305.19923 ↗First posted 31 May 2023cs.LG cs.AIcited by 65

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Recently, diffusion model shines as a promising backbone for the sequence modeling paradigm in offline reinforcement learning(RL). However, these works mostly lack the generalization ability across tasks with reward or dynamics change. To tackle this challenge, in this paper we propose a task-oriented conditioned diffusion planner for offline meta-RL(MetaDiffuser), which considers the generalization problem as conditional trajectory generation task with contextual representation. The key is to learn a context conditioned diffusion model which can generate task-oriented trajectories for planning across diverse tasks. To enhance the dynamics consistency of the generated trajectories while encouraging trajectories to achieve high returns, we further design a dual-guided module in the sampling process of the diffusion model. The proposed framework enjoys the robustness to the quality of collected warm-start data from the testing task and the flexibility to incorporate with different task representation method. The experiment results on MuJoCo benchmarks show that MetaDiffuser outperforms other strong offline meta-RL baselines, demonstrating the outstanding conditional generation ability of diffusion architecture.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 36 of its 45 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

36 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov
  2. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning12 Aug

Other references · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
22 more
  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2018Group Normalizationcited by 1,514
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  7. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  8. 2021Scaling Vision Transformerscited by 993
  9. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  10. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  11. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  12. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  13. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  14. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  15. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 143
  16. 2019Mish: A Self Regularized Non-Monotonic Activation Functioncited by 141
  17. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
  18. 2022Bootstrapped Transformer for Offline Reinforcement Learningcited by 37
  19. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 33
  20. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  21. 2020Dynamic Knapsack Optimization Towards Efficient Multi-Channel Sequential Advertisingcited by 16
  22. 2022EUCLID: Towards Efficient Unsupervised Reinforcement Learning with Multi-choice Dynamics Modelcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 31; 2025: 19; 2026: 9.

65 papers.

Sort:

By area:

  1. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
  2. 2024Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimizationcited by 99
  3. 2025Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policiescited by 90
  4. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 84
  5. 2023SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Executioncited by 69
  6. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 68
  7. 2024ManiCM: Real-time 3D Diffusion Policy via Consistency Model for Robotic Manipulationcited by 57
  8. 2024DiffuserLite: Towards Real-time Diffusion Planningcited by 40
  9. 2024CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Makingcited by 38
  10. 2023AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Modelcited by 37
  11. 2024G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulationcited by 35
  12. 2024Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Trainingcited by 26
38 more
  1. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  2. 2024AdvDiffuser: Generating Adversarial Safety-Critical Driving Scenarios via Guided Diffusioncited by 20
  3. 2024Diffusion Model for Planning: A Systematic Literature Reviewcited by 17
  4. 2024Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglementcited by 16
  5. 2025GenPO: Generative Diffusion Models Meet On-Policy Reinforcement Learningcited by 16
  6. 2024Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimizationcited by 15
  7. 2024DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulationcited by 15
  8. 2024Resisting Stochastic Risks in Diffusion Planners with the Trajectory Aggregation Treecited by 13
  9. 2023Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learningcited by 12
  10. 2024Aligning Diffusion Behaviors with Q-functions for Efficient Continuous Controlcited by 12
  11. 2023Efficient Planning with Latent Diffusioncited by 11
  12. 2025State-Covering Trajectory Stitching for Diffusion Plannerscited by 11
  13. 2024Forward KL Regularized Preference Optimization for Aligning Diffusion Policiescited by 10
  14. 2023Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitationscited by 9
  15. 2024Regularized Conditional Diffusion Model for Multi-Task Preference Alignmentcited by 9
  16. 2024PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learningcited by 9
  17. 2025Multi-agent Coordination via Flow Matchingcited by 9
  18. 2024Learning from Random Demonstrations: Offline Reinforcement Learning with Importance-Sampled Diffusion Modelscited by 8
  19. 2024Diffusion Models as Optimizers for Efficient Planning in Offline RLcited by 8
  20. 2025Local Manifold Approximation and Projection for Manifold-Aware Diffusion Planningcited by 8
  21. 2025DmC: Nearest Neighbor Guidance Diffusion Model for Offline Cross-domain Reinforcement Learningcited by 7
  22. 2025Towards Robust Zero-Shot Reinforcement Learningcited by 7
  23. 2024Continual Offline Reinforcement Learning via Diffusion-based Dual Generative Replaycited by 6
  24. 2024Prompt Tuning with Diffusion for Few-Shot Pre-trained Policy Generalizationcited by 6
  25. 2024Stable Continual Reinforcement Learning via Diffusion-based Trajectory Replaycited by 6
  26. 2024The Duality of Generative AI and Reinforcement Learning in Robotics: A Reviewcited by 5
  27. 2025Provable Sample-Efficient Transfer Learning Conditional Diffusion Models via Representation Learningcited by 5
  28. 2025Text-to-Decision Agent: Offline Meta-Reinforcement Learning from Natural Language Supervisioncited by 5
  29. 2025Scalable In-Context Q-Learningcited by 5
  30. 2025Generative Auto-Bidding in Large-Scale Competitive Auctions via Diffusion Completer-Alignercited by 5
  31. 2024MODULI: Unlocking Preference Generalization via Diffusion Models for Offline Multi-Objective Reinforcement Learningcited by 4
  32. 2024Hierarchical Prompt Decision Transformer: Improving Few-Shot Policy Generalization with Global and Adaptive Guidancecited by 3
  33. 2025EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modelingcited by 3
  34. 2025SwarmDiff: Swarm Robotic Trajectory Planning in Cluttered Environments via Diffusion Transformercited by 3
  35. 2025VER: Vision Expert Transformer for Robot Learning via Foundation Distillation and Dynamic Routingcited by 3
  36. 2024Disentangling Policy from Offline Task Representation Learning via Adversarial Data Augmentationcited by 2
  37. 2024Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsalcited by 2
  38. 2025HyCodePolicy: Hybrid Language Controllers for Multimodal Monitoring and Decision in Embodied Agentscited by 2

50 most cited of 65 All 65 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Stitching Sub-Trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL
  2. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning
  3. 2024Task-Agnostic Pre-training and Task-Guided Fine-tuning for Versatile Diffusion Planner
  4. 2024Simple Hierarchical Planning with Diffusion
  5. 2023Efficient Diffusion Policies for Offline Reinforcement Learning
  6. 2020FOCAL: Efficient Fully-Offline Meta-Reinforcement Learning via Distance Metric Learning and Behavior Regularization
  7. 2024Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learning
  8. 2025Accelerating Diffusion Planners in Offline RL via Reward-Aware Consistency Trajectory Distillation