# Diffusion Model-Augmented Behavioral Cloning

arXiv: https://arxiv.org/abs/2302.13335 · HTML: https://mlsanity.com/paper/2302.13335/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 45 of its 80 references that are in the map.
- First posted on arXiv 26 Feb 2023.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)
- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 30 of 45)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Implicit Behavioral Cloning](https://mlsanity.com/paper/2109.00137/) (2021)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [Exploring the Limitations of Behavior Cloning for Autonomous Driving](https://mlsanity.com/paper/1904.08980/) (2019)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [Behavior Transformers: Cloning k modes with one stone](https://mlsanity.com/paper/2206.11251/) (2022)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Imitating Human Behaviour with Diffusion Models](https://mlsanity.com/paper/2301.10677/) (2023)

## Built on by (most cited first, 30 of 38)

- [Learning to Act from Actionless Videos through Dense Correspondences](https://mlsanity.com/paper/2310.08576/) (2023)
- [Parallel Sampling of Diffusion Models](https://mlsanity.com/paper/2305.16317/) (2023)
- [The Ingredients for Robotic Diffusion Transformers](https://mlsanity.com/paper/2410.10088/) (2024)
- [RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learning](https://mlsanity.com/paper/2510.14830/) (2025)
- [Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimization](https://mlsanity.com/paper/2308.05384/) (2023)
- [DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasets](https://mlsanity.com/paper/2404.19264/) (2024)
- [Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots](https://mlsanity.com/paper/2405.01792/) (2024)
- [Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learning](https://mlsanity.com/paper/2307.01849/) (2023)
- [Boosting Continuous Control with Consistency Policy](https://mlsanity.com/paper/2310.06343/) (2023)
- [Diffusion Model Predictive Control](https://mlsanity.com/paper/2410.05364/) (2024)
- [World-Gymnast: Training Robots with Reinforcement Learning in a World Model](https://mlsanity.com/paper/2602.02454/) (2026)
- [VideoAgent: Self-Improving Video Generation](https://mlsanity.com/paper/2410.10076/) (2024)
- [Dexterous Manipulation through Imitation Learning: A Survey](https://mlsanity.com/paper/2504.03515/) (2025)
- [Diffusion-Reward Adversarial Imitation Learning](https://mlsanity.com/paper/2405.16194/) (2024)
- [Fast and Robust Visuomotor Riemannian Flow Matching Policy](https://mlsanity.com/paper/2412.10855/) (2024)
- [One ACT Play: Single Demonstration Behavior Cloning with Action Chunking Transformers](https://mlsanity.com/paper/2309.10175/) (2023)
- [Diffusing States and Matching Scores: A New Framework for Imitation Learning](https://mlsanity.com/paper/2410.13855/) (2024)
- [Falcon: Fast Visuomotor Policies via Partial Denoising](https://mlsanity.com/paper/2503.00339/) (2025)
- [Pixel Motion as Universal Representation for Robot Control](https://mlsanity.com/paper/2505.07817/) (2025)
- [Diffusion Imitation from Observation](https://mlsanity.com/paper/2410.05429/) (2024)
- [Demystifying Diffusion Policies: Action Memorization and Simple Lookup Table Alternatives](https://mlsanity.com/paper/2505.05787/) (2025)
- [Score-Based Diffusion Policy Compatible with Reinforcement Learning via Optimal Transport](https://mlsanity.com/paper/2502.12631/) (2025)
- [Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations](https://mlsanity.com/paper/2512.21586/) (2025)
- [WARPD: World model Assisted Reactive Policy Diffusion](https://mlsanity.com/paper/2410.14040/) (2024)
- [Imitation Learning from Purified Demonstrations](https://mlsanity.com/paper/2310.07143/) (2023)
- [Toward Open-ended Embodied Tasks Solving](https://mlsanity.com/paper/2312.05822/) (2023)
- [SAD-Flower: Flow Matching for Safe, Admissible, and Dynamically Consistent Planning](https://mlsanity.com/paper/2511.05355/) (2025)
- [Augmenting Offline Reinforcement Learning with State-only Interactions](https://mlsanity.com/paper/2402.00807/) (2024)
- [Dual-Granularity Contrastive Reward via Generated Episodic Guidance for Efficient Embodied RL](https://mlsanity.com/paper/2602.12636/) (2026)
- [Program Machine Policy: Addressing Long-Horizon Tasks by Integrating Program Synthesis and State Machines](https://mlsanity.com/paper/2311.15960/) (2023)

## Abstract (arXiv)

Imitation learning addresses the challenge of learning by observing an expert's demonstrations without access to reward signals from environments. Most existing imitation learning methods that do not require interacting with environments either model the expert distribution as the conditional probability p(a|s) (e.g., behavioral cloning, BC) or the joint probability p(s, a). Despite the simplicity of modeling the conditional probability with BC, it usually struggles with generalization. While modeling the joint probability can improve generalization performance, the inference procedure is often time-consuming, and the model can suffer from manifold overfitting. This work proposes an imitation learning framework that benefits from modeling both the conditional and joint probability of the expert distribution. Our proposed Diffusion Model-Augmented Behavioral Cloning (DBC) employs a diffusion model trained to model expert behaviors and learns a policy to optimize both the BC loss (conditional) and our proposed diffusion model loss (joint). DBC outperforms baselines in various continuous control tasks in navigation, robot arm manipulation, dexterous manipulation, and locomotion. We design additional experiments to verify the limitations of modeling either the conditional probability or the joint probability of the expert distribution, as well as compare different generative models. Ablation studies justify the effectiveness of our design choices.