mlsanityData to 7 Oct 2026

Generative predecessor models for sample-efficient imitation learning

arXiv 1904.01139 ↗First posted 1 Apr 2019cs.LG stat.MLcited by 14

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We propose Generative Predecessor Models for Imitation Learning (GPRIL), a novel imitation learning algorithm that matches the state-action distribution to the distribution observed in expert demonstrations, using generative models to reason probabilistically about alternative histories of demonstrated states. We show that this approach allows an agent to learn robust policies using only a small number of expert demonstrations and self-supervised interactions with the environment. We derive this approach from first principles and compare it empirically to a state-of-the-art imitation learning method, showing that it outperforms or matches its performance on two simulated robot manipulation tasks and demonstrate significantly higher sample efficiency by applying the algorithm on a real robot.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 38 references that are in the map. First posted on arXiv 1 Apr 2019.

Builds on

19 of 38 references.

References · 19

  1. 2013Auto-Encoding Variational Bayescited by 3,980
  2. 2014Conditional Generative Adversarial Netscited by 2,152
  3. 2016Density estimation using Real NVPcited by 2,029
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2016Pixel Recurrent Neural Networkscited by 814
  7. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  8. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
  9. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  10. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  11. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  12. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
7 more
  1. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  2. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  3. 2018Zero-Shot Visual Imitationcited by 114
  4. 2018Playing hard exploration games by watching YouTubecited by 98
  5. 2018Recall Traces: Backtracking Models for Efficient Reinforcement Learningcited by 23
  6. 2018Organizing Experience: A Deeper Look at Replay Mechanisms for Sample-based Planning in Continuous State Domainscited by 19
  7. 2018Forward-Backward Reinforcement Learningcited by 17

Built on by

Cited by year

Citing papers per year: 2019: 2; 2020: 3; 2021: 1; 2022: 6; 2023: 1; 2024: 1.

14 papers.

Sort:

By area:

  1. 2019Goal-conditioned Imitation Learningcited by 159
  2. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 92
  3. 2020Intrinsic Reward Driven Imitation Learning via Generative Modelcited by 35
  4. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  5. 2022Topological Experience Replaycited by 17
  6. 2019VILD: Variational Imitation Learning with Diverse-quality Demonstrationscited by 12
  7. 2020Universal Value Density Estimation for Imitation Learning and Goal-Conditioned Reinforcement Learningcited by 12
  8. 2023A Coupled Flow Approach to Imitation Learningcited by 12
  9. 2022Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulationcited by 7
  10. 2022Robust Imitation of a Few Demonstrations with a Backwards Modelcited by 7
  11. 2022Imitation Learning by State-Only Distribution Matchingcited by 6
  12. 2022Reward-Free Policy Space Compression for Reinforcement Learningcited by 6
2 more
  1. 2020Support-weighted Adversarial Imitation Learningcited by 3
  2. 2022Introspective Experience Replay: Look Back When Surprisedcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2022Diverse Imitation Learning via Self-Organizing Generative Models
  2. 2022Robust Learning from Observation with Model Misspecification
  3. 2022Imitation Learning by Estimating Expertise of Demonstrators
  4. 2021CRIL: Continual Robot Imitation Learning via Generative and Prediction Model
  5. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
  6. 2019A New Framework for Query Efficient Active Imitation Learning
  7. 2024Efficient Imitation Learning with Conservative World Models
  8. 2022Model-Based Imitation Learning Using Entropy Regularization of Model and Policy