mlsanityData to 7 Oct 2026

Diffusion Imitation from Observation

arXiv 2410.05429 ↗First posted 7 Oct 2024cs.LGcited by 12

On the map: Diffusion models → Diffusion and flow policies for robotics

also in 1 more
Abstract (arXiv)

Learning from observation (LfO) aims to imitate experts by learning from state-only demonstrations without requiring action labels. Existing adversarial imitation learning approaches learn a generator agent policy to produce state transitions that are indistinguishable to a discriminator that learns to classify agent and expert state transitions. Despite its simplicity in formulation, these methods are often sensitive to hyperparameters and brittle to train. Motivated by the recent success of diffusion models in generative modeling, we propose to integrate a diffusion model into the adversarial imitation learning from observation framework. Specifically, we employ a diffusion model to capture expert and agent transitions by generating the next state, given the current state. Then, we reformulate the learning objective to train the diffusion model as a binary classifier and use it to provide "realness" rewards for policy learning. Our proposed framework, Diffusion Imitation from Observation (DIFO), demonstrates superior performance in various continuous control domains, including navigation, locomotion, manipulation, and games. Project page: https://nturobotlearninglab.github.io/DIFO

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 61 of its 84 references that are in the map. First posted on arXiv 7 Oct 2024.

Builds on

61 of 84 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Diffusion-Reward Adversarial Imitation Learning25 May
  2. 2023Diffusion Reward: Learning Rewards via Conditional Video Diffusion21 Dec
  3. 2023DiffAIL: Diffusion Adversarial Imitation Learning11 Dec
  4. 2023Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models13 Oct
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
  6. 2023Diffusion Model-Augmented Behavioral Cloning26 Feb
  7. 2023Imitating Human Behaviour with Diffusion Models25 Jan

Other references · 54

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2016Generative Adversarial Imitation Learningcited by 1,576
  11. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
38 more
  1. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  2. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  3. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  4. 2018Behavioral Cloning from Observationcited by 482
  5. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  6. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  7. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  8. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  9. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 301
  10. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  11. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  12. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  13. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  14. 2018Generative Adversarial Imitation from Observationcited by 183
  15. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 180
  16. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 159
  17. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  18. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  19. 2023Learning to Act without Actionscited by 116
  20. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  21. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 92
  22. 2020Reinforcement Learning with Videos: Combining Offline Observations with Interactioncited by 80
  23. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  24. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 76
  25. 2021Off-Policy Imitation Learning from Observationscited by 70
  26. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 64
  27. 2019Wasserstein Adversarial Imitation Learningcited by 59
  28. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
  29. 2020TrajGAIL: Generating Urban Vehicle Trajectories using Generative Adversarial Imitation Learningcited by 46
  30. 2022DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANscited by 43
  31. 2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
  32. 2017OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learningcited by 40
  33. 2016Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstrationcited by 35
  34. 2022imitation: Clean Imitation Learning Implementationscited by 34
  35. 2023Robotic Offline RL from Internet Videos via Value-Function Pre-Trainingcited by 31
  36. 2020Imitation Learning with Sinkhorn Distancescited by 30
  37. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimizationcited by 22
  38. 2018Imitation Learning with Concurrent Actions in 3D Gamescited by 18

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 4.

12 papers.

  1. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learningcited by 16
  2. 2025SMP: Reusable Score-Matching Motion Priors for Physics-Based Character Controlcited by 14
  3. 2025Action-Constrained Imitation Learningcited by 2
  4. 2025SAD-Flower: Flow Matching for Safe, Admissible, and Dynamically Consistent Planningcited by 2
  5. 2025Noise-conditioned Energy-based Annealed Rewards (NEAR): A Generative Framework for Imitation Learning from Observationcited by 1
  6. 2025On Generalization and Distributional Update for Mimicking Observations with Adequate Explorationcited by 0
  7. 2025CooT: Learning to Coordinate In-Context with Coordination Transformerscited by 0
  8. 2025Learning from Observation: A Survey of Recent Advancescited by 0
  9. 2026Latent Wasserstein Adversarial Imitation Learningcited by 0
  10. 2026Language-Critique Imitation Learning from Suboptimal Demonstrationscited by 0
  11. 2026Label-free Industrial Fault Detection via Adversarial Inverse Reinforcement Learning: A System for Run-to-Failure Prognosticscited by 0
  12. 2026Loop-Free Inverse Reinforcement Learning via Sequential Value Recovery with Q-Score Matchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Imitation Learning from Observations under Transition Model Disparity
  2. 2022Imitation Learning by State-Only Distribution Matching
  3. 2023Adversarial Imitation Learning from Visual Observations using Latent Information
  4. 2024Don't Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion
  5. 2025SD2AIL: Adversarial Imitation Learning from Synthetic Demonstrations via Diffusion Models
  6. 2022Adversarial Imitation Learning from Video using a State Observer
  7. 2020State-only Imitation with Transition Dynamics Mismatch
  8. 2023Imitation Learning from Observation through Optimal Transport