mlsanityData to 14 Oct 2026

Diffusion-Reward Adversarial Imitation Learning

arXiv 2405.16194 ↗First posted 25 May 2024cs.LG cs.AI cs.ROcited by 20

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Imitation learning aims to learn a policy from observing expert demonstrations without access to reward signals from environments. Generative adversarial imitation learning (GAIL) formulates imitation learning as adversarial learning, employing a generator policy learning to imitate expert behaviors and discriminator learning to distinguish the expert demonstrations from agent trajectories. Despite its encouraging results, GAIL training is often brittle and unstable. Inspired by the recent dominance of diffusion models in generative modeling, we propose Diffusion-Reward Adversarial Imitation Learning (DRAIL), which integrates a diffusion model into GAIL, aiming to yield more robust and smoother rewards for policy learning. Specifically, we propose a diffusion discriminative classifier to construct an enhanced discriminator, and design diffusion rewards based on the classifier's output for policy learning. Extensive experiments are conducted in navigation, manipulation, and locomotion, verifying DRAIL's effectiveness compared to prior imitation learning methods. Moreover, additional experimental results demonstrate the generalizability and data efficiency of DRAIL. Visualized learned reward functions of GAIL and DRAIL suggest that DRAIL can produce more robust and smoother rewards. Project page: https://nturobotlearninglab.github.io/DRAIL/

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 40 of its 60 references that are in the map. First posted on arXiv 25 May 2024.

Builds on

40 of 60 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023DiffAIL: Diffusion Adversarial Imitation Learning11 Dec
  2. 2023Extracting Reward Functions from Diffusion Models1 Jun
  3. 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policies5 Apr
  4. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  5. 2023Diffusion Model-Augmented Behavioral Cloning26 Feb
  6. 2023Imitating Human Behaviour with Diffusion Models25 Jan

Other references · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  10. 2016Generative Adversarial Imitation Learningcited by 1,597
  11. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  12. 2016Variational Inference: A Review for Statisticianscited by 1,472
22 more
  1. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  2. 2021Variational Diffusion Modelscited by 1,153
  3. 2017Deep Reinforcement Learning that Matterscited by 754
  4. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  5. 2018Behavioral Cloning from Observationcited by 492
  6. 2018An Algorithmic Perspective on Imitation Learningcited by 459
  7. 2021Implicit Behavioral Cloningcited by 452
  8. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  9. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 306
  10. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  11. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 204
  12. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 193
  13. 2018Generative Adversarial Imitation from Observationcited by 184
  14. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  15. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 164
  16. 2020Primal Wasserstein Imitation Learningcited by 121
  17. 2018Playing hard exploration games by watching YouTubecited by 99
  18. 2021What Matters for Adversarial Imitation Learning?cited by 70
  19. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  20. 2023Inverse Reinforcement Learning without Reinforcement Learningcited by 36
  21. 2018Imitation Learning with Concurrent Actions in 3D Gamescited by 18
  22. 2023A Coupled Flow Approach to Imitation Learningcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 10; 2026: 6.

20 papers.

Sort:

By area:

  1. 2024The Ingredients for Robotic Diffusion Transformerscited by 73
  2. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learningcited by 16
  3. 2024Diffusion Imitation from Observationcited by 13
  4. 2025NIL: No-data Imitation Learning by Leveraging Pre-trained Video Diffusion Modelscited by 5
  5. 2025MisoDICE: Multi-Agent Imitation from Unlabeled Mixed-Quality Demonstrationscited by 3
  6. 2025Latent Conditioned Loco-Manipulation Using Motion Priorscited by 3
  7. 2025SAD-Flower: Flow Matching for Safe, Admissible, and Dynamically Consistent Planningcited by 3
  8. 2025Action-Constrained Imitation Learningcited by 2
  9. 2024RILe: Reinforced Imitation Learningcited by 1
  10. 2025On Discovering Algorithms for Adversarial Imitation Learningcited by 1
  11. 2025Restoring Noisy Demonstration for Imitation Learning With Diffusion Modelscited by 1
  12. 2026Constraint-Aware Diffusion Priors for High-Fidelity and Versatile Quadruped Locomotioncited by 1
8 more
  1. 2025Diffusion Classifier-Driven Reward for Offline Preference-based Reinforcement Learningcited by 0
  2. 2025CooT: Learning to Coordinate In-Context with Coordination Transformerscited by 0
  3. 2025SD2AIL: Adversarial Imitation Learning from Synthetic Demonstrations via Diffusion Modelscited by 0
  4. 2026TacSIm: A Dataset and Benchmark for Football Tactical Style Imitationcited by 0
  5. 2026Recovering Hidden Reward in Diffusion-Based Policiescited by 0
  6. 2026Adversarial Dual On-Policy Distillation from Expressive Teachercited by 0
  7. 2026Language-Critique Imitation Learning from Suboptimal Demonstrationscited by 0
  8. 2026Loop-Free Inverse Reinforcement Learning via Sequential Value Recovery with Q-Score Matchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020On Computation and Generalization of Generative Adversarial Imitation Learning
  2. 2019Adversarial Imitation Learning from Incomplete Demonstrations
  3. 2022Latent Policies for Adversarial Imitation Learning
  4. 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective
  5. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
  6. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
  7. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
  8. 2020When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence