mlsanityData to 14 Oct 2026

Good Better Best: Self-Motivated Imitation Learning for noisy Demonstrations

arXiv 2310.15815 ↗First posted 24 Oct 2023cs.LGcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation Learning (IL) aims to discover a policy by minimizing the discrepancy between the agent's behavior and expert demonstrations. However, IL is susceptible to limitations imposed by noisy demonstrations from non-expert behaviors, presenting a significant challenge due to the lack of supplementary information to assess their expertise. In this paper, we introduce Self-Motivated Imitation LEarning (SMILE), a method capable of progressively filtering out demonstrations collected by policies deemed inferior to the current policy, eliminating the need for additional information. We utilize the forward and reverse processes of Diffusion Models to emulate the shift in demonstration expertise from low to high and vice versa, thereby extracting the noise information that diffuses expertise. Then, the noise information is leveraged to predict the diffusion steps between the current policy and demonstrators, which we theoretically demonstrate its equivalence to their expertise gap. We further explain in detail how the predicted diffusion steps are applied to filter out noisy demonstrations in a self-motivated manner and provide its theoretical grounds. Through empirical evaluations on MuJoCo tasks, we demonstrate that our method is proficient in learning the expert policy amidst noisy demonstrations, and effectively filters out demonstrations with expertise inferior to the current policy.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 29 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

17 of 29 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Imitation Learning by Estimating Expertise of Demonstrators2 Feb
  2. 2020Robust Imitation Learning from Noisy Demonstrations20 Oct
  3. 2020Energy-Based Imitation Learning20 Apr
  4. 2019Imitation Learning from Imperfect Demonstration27 Jan

Other references · 13

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  5. 2016Generative Adversarial Imitation Learningcited by 1,597
  6. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  7. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  8. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  9. 2023Imitating Human Behaviour with Diffusion Modelscited by 263
  10. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 260
  11. 2020Learning Energy-Based Models by Diffusion Recovery Likelihoodcited by 123
  12. 2017Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximationcited by 121
1 more
  1. 2015Document Context Language Modelscited by 10

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  2. 2024DIDA: Denoised Imitation Learning based on Domain Adaptationcited by 2
  3. 2025Demonstration Sidetracks: Categorizing Systematic Non-Optimality in Human Demonstrationscited by 2
  4. 2025ACG: Action Coherence Guidance for Flow-based Vision-Language-Action modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Restoring Noisy Demonstration for Imitation Learning With Diffusion Models
  2. 2023Imitation Learning from Purified Demonstrations
  3. 2023Skill Disentanglement for Imitation Learning from Suboptimal Demonstrations
  4. 2024Imitation Learning from Suboptimal Demonstrations via Meta-Learning An Action Ranker
  5. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality
  6. 2022Diverse Imitation Learning via Self-Organizing Generative Models
  7. 2024Don't Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusion
  8. 2021Imitation Learning: Progress, Taxonomies and Challenges