mlsanityData to 7 Oct 2026

Goal-Conditioned Imitation Learning using Score-based Diffusion Policies

arXiv 2304.02532 ↗First posted 5 Apr 2023cs.LG cs.ROcited by 229

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

We propose a new policy representation based on score-based diffusion models (SDMs). We apply our new policy representation in the domain of Goal-Conditioned Imitation Learning (GCIL) to learn general-purpose goal-specified policies from large uncurated datasets without rewards. Our new goal-conditioned policy architecture "BEhavior generation with ScOre-based Diffusion Policies" (BESO) leverages a generative, score-based diffusion model as its policy. BESO decouples the learning of the score model from the inference sampling process, and, hence allows for fast sampling strategies to generate goal-specified behavior in just 3 denoising steps, compared to 30+ steps of other diffusion based policies. Furthermore, BESO is highly expressive and can effectively capture multi-modality present in the solution space of the play data. Unlike previous methods such as Latent Plans or C-Bet, BESO does not rely on complex hierarchical policies or additional clustering for effective goal-conditioned behavior learning. Finally, we show how BESO can even be used to learn a goal-independent policy from play-data using classifier-free guidance. To the best of our knowledge this is the first work that a) represents a behavior policy based on such a decoupled SDM b) learns an SDM based policy in the domain of GCIL and c) provides a way to simultaneously learn a goal-dependent and a goal-independent policy from play-data. We evaluate BESO through detailed simulation and show that it consistently outperforms several state-of-the-art goal-conditioned imitation learning methods on challenging benchmarks. We additionally provide extensive ablation studies and experiments to demonstrate the effectiveness of our method for goal-conditioned behavior generation. Demonstrations and Code are available at https://intuitive-robots.github.io/beso-website/

Key facts. Cited by 229 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 46 of its 58 references that are in the map. First posted on arXiv 5 Apr 2023.

Builds on

46 of 58 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  2. 2023Imitating Human Behaviour with Diffusion Models25 Jan
  3. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov
  4. 2022SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusion8 Sep
  5. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning12 Aug

Other references · 41

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2018Neural Ordinary Differential Equationscited by 3,733
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
29 more
  1. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  2. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  3. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2022Human Motion Diffusion Modelcited by 1,062
  6. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  7. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  8. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  9. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  10. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  11. 2021Implicit Behavioral Cloningcited by 442
  12. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  13. 2019Learning Latent Plans from Playcited by 310
  14. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  15. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  16. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  17. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 137
  18. 2020Parrot: Data-Driven Behavioral Priors for Reinforcement Learningcited by 136
  19. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrationscited by 114
  20. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  21. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  22. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  23. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regressioncited by 65
  24. 2020Energy-Based Imitation Learningcited by 39
  25. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  26. 2022Imitating Past Successes can be Very Suboptimalcited by 20
  27. 2021Playful Interactions for Representation Learningcited by 13
  28. 2022PLATO: Predicting Latent Affordances Through Object-Centric Playcited by 12
  29. 2020Imitation with Neural Density Modelscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 20; 2024: 78; 2025: 80; 2026: 51.

229 papers.

Sort:

By area:

  1. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  2. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  3. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  4. 2024Diffusion Policy Policy Optimizationcited by 255
  5. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 234
  6. 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
  7. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  8. 2024Behavior Generation with Latent Actionscited by 159
  9. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 159
  10. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 131
  11. 2023Learning a Diffusion Model Policy from Rewards via Q-Score Matchingcited by 126
  12. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
38 more
  1. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  2. 2024Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimizationcited by 99
  3. 2024FlowPolicy: Enabling Fast and Robust 3D Flow-based Policy via Consistency Flow Matching for Robot Manipulationcited by 99
  4. 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learningcited by 93
  5. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 84
  6. 2024One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillationcited by 83
  7. 2025ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learningcited by 83
  8. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 80
  9. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  10. 2023MADiff: Offline Multi-agent Learning with Diffusion Modelscited by 73
  11. 2025Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policycited by 73
  12. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  13. 2024The Ingredients for Robotic Diffusion Transformerscited by 72
  14. 2024Riemannian Flow Matching Policy for Robot Motion Learningcited by 61
  15. 2024Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theorycited by 58
  16. 2024ManiCM: Real-time 3D Diffusion Policy via Consistency Model for Robotic Manipulationcited by 57
  17. 2023Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimizationcited by 56
  18. 2025FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policiescited by 56
  19. 2023Imitation Bootstrapped Reinforcement Learningcited by 54
  20. 2024Affordance-based Robot Manipulation with Flow Matchingcited by 54
  21. 2025DIME:Diffusion-Based Maximum Entropy Reinforcement Learningcited by 52
  22. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrationscited by 51
  23. 2024Streaming Diffusion Policy: Fast Policy Synthesis with Variable Noise Diffusion Modelscited by 51
  24. 2024Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradientcited by 49
  25. 2024Efficient Diffusion Transformer Policies with Mixture of Expert Denoisers for Multitask Learningcited by 46
  26. 2024Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learningcited by 44
  27. 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 43
  28. 2024PoCo: Policy Composition from and for Heterogeneous Robot Learningcited by 42
  29. 2024Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Followingcited by 41
  30. 2024RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketchescited by 41
  31. 2024Diffusion Transformer Policycited by 41
  32. 2023Movement Primitive Diffusion: Learning Gentle Robotic Manipulation of Deformable Objectscited by 40
  33. 2025Diffusion Models for Robotic Manipulation: A Surveycited by 39
  34. 2025DynaGuide: Steering Diffusion Polices with Active Dynamic Guidancecited by 39
  35. 2024Scaling Diffusion Policy in Transformer to 1 Billion Parameters for Robotic Manipulationcited by 38
  36. 2023Boosting Continuous Control with Consistency Policycited by 36
  37. 2024JUICER: Data-Efficient Imitation Learning for Robotic Assemblycited by 36
  38. 2024Efficient Data Collection for Robotic Manipulation via Compositional Generalizationcited by 35

50 most cited of 229 All 229 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Efficient Online Reinforcement Learning for Diffusion Policy
  2. 2025Consistent Zero-Shot Imitation with Contrastive Goal Inference
  3. 2026Training-Free Imitation Learning with Closed-Form Diffusion Policies
  4. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learning
  5. 2023Score Regularized Policy Optimization through Diffusion Behavior
  6. 2025Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning
  7. 2024Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning
  8. 2022Diverse Imitation Learning via Self-Organizing Generative Models