mlsanityData to 7 Oct 2026

Provably Efficient Imitation Learning from Observation Alone

arXiv 1905.10948 ↗First posted 27 May 2019cs.LG stat.MLcited by 78

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We study Imitation Learning (IL) from Observations alone (ILFO) in large-scale MDPs. While most IL algorithms rely on an expert to directly provide actions to the learner, in this setting the expert only supplies sequences of observations. We design a new model-free algorithm for ILFO, Forward Adversarial Imitation Learning (FAIL), which learns a sequence of time-dependent policies by minimizing an Integral Probability Metric between the observation distributions of the expert policy and the learner. FAIL is the first provably efficient algorithm in ILFO setting, which learns a near-optimal policy with a number of samples that is polynomial in all relevant parameters but independent of the number of unique observations. The resulting theory extends the domain of provably sample efficient learning algorithms beyond existing results, which typically only consider tabular reinforcement learning settings or settings that require access to a near-optimal reset distribution. We also investigate the extension of FAIL in a model-based setting. Finally we demonstrate the efficacy of FAIL on multiple OpenAI Gym control tasks.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 21 of its 51 references that are in the map. First posted on arXiv 27 May 2019.

Builds on

21 of 51 references.

Closest ancestors · 3

Same direction, newest first

  1. 2018Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning29 May
  2. 2018Imitating Latent Policies from Observation21 May
  3. 2018Behavioral Cloning from Observation4 May

Other references · 18

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2016OpenAI Gymcited by 1,684
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2017Wasserstein GANcited by 635
  6. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
  7. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 285
  8. 2014Taming the Monster: A Fast and Simple Algorithm for Contextual Banditscited by 200
  9. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  10. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 165
  11. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 116
  12. 2015Sample Complexity of Episodic Fixed-Horizon Reinforcement Learningcited by 105
6 more
  1. 2017Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Predictioncited by 93
  2. 2017Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulationcited by 90
  3. 2015Learning to Search Better Than Your Teachercited by 50
  4. 2015Learning to Filter with Predictive State Inference Machinescited by 8
  5. 2018Model-based RL in Contextual Decision Processes: PAC bounds and Exponential Improvements over Model-free Approachescited by 5
  6. 2015Learning to Search for Dependenciescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 4; 2020: 6; 2021: 17; 2022: 16; 2023: 9; 2024: 13; 2025: 6; 2026: 7.

78 papers.

Sort:

By area:

  1. 2022Offline Reinforcement Learning with Realizability and Single-policy Concentrabilitycited by 122
  2. 2019Imitation Learning as f-Divergence Minimizationcited by 121
  3. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 101
  4. 2020Toward the Fundamental Limits of Imitation Learningcited by 96
  5. 2019Recent Advances in Imitation Learning from Observationcited by 92
  6. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 90
  7. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gapcited by 80
  8. 2019State Alignment-based Imitation Learningcited by 73
  9. 2021Off-Policy Imitation Learning from Observationscited by 70
  10. 2021Feedback in Imitation Learning: The Three Regimes of Covariate Shiftcited by 68
  11. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
  12. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
38 more
  1. 2021MobILE: Model-Based Imitation Learning From Observation Alonecited by 44
  2. 2020Provable Representation Learning for Imitation Learning via Bi-level Optimizationcited by 41
  3. 2020Energy-Based Imitation Learningcited by 39
  4. 2023Inverse Reinforcement Learning without Reinforcement Learningcited by 35
  5. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
  6. 2021A Simple Reward-free Approach to Constrained Reinforcement Learningcited by 28
  7. 2022Causal Imitation Learning under Temporally Correlated Noisecited by 28
  8. 2020Robust Imitation Learning from Noisy Demonstrationscited by 26
  9. 2021Interactive Learning from Activity Descriptioncited by 26
  10. 2021Learning from Imperfect Demonstrations from Agents with Varying Dynamicscited by 26
  11. 2021Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coveragecited by 26
  12. 2022LobsDICE: Offline Learning from Observation via Stationary Distribution Correction Estimationcited by 22
  13. 2022A Ranking Game for Imitation Learningcited by 18
  14. 2021Imitation Learning by Reinforcement Learningcited by 17
  15. 2022SelfD: Self-Learning Large-Scale Driving Policies From the Webcited by 16
  16. 2023Identifiability and Generalizability in Constrained Inverse Reinforcement Learningcited by 16
  17. 2024A Survey of Imitation Learning Methods, Environments and Metricscited by 16
  18. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learningcited by 16
  19. 2023What Matters to You? Towards Visual Representation Alignment for Robot Learningcited by 14
  20. 2021Imitation by Predicting Observationscited by 13
  21. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  22. 2022Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysiscited by 13
  23. 2021Provably Breaking the Quadratic Error Compounding Barrier in Imitation Learning, Optimallycited by 12
  24. 2023The Virtues of Laziness in Model-based RL: A Unified Objective and Algorithmscited by 12
  25. 2022Imitation Learning from Observations under Transition Model Disparitycited by 11
  26. 2021Learning Feasibility to Imitate Demonstrators with Different Dynamicscited by 10
  27. 2024A Dual Approach to Imitation Learning from Observations with Offline Datasetscited by 10
  28. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matchingcited by 10
  29. 2024MILES: Making Imitation Learning Easy with Self-Supervisioncited by 9
  30. 2023Imitation Learning from Observation through Optimal Transportcited by 7
  31. 2023Offline Imitation Learning with Variational Counterfactual Reasoningcited by 7
  32. 2025Imitation Learning from a Single Temporally Misaligned Videocited by 7
  33. 2022Imitation Learning by State-Only Distribution Matchingcited by 6
  34. 2022Improved Policy Optimization for Online Imitation Learningcited by 6
  35. 2022Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrationscited by 6
  36. 2022Model Predictive Control via On-Policy Imitation Learningcited by 6
  37. 2023Learning non-Markovian Decision-Making from State-only Sequencescited by 6
  38. 2024Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximationcited by 6

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions
  2. 2019Sample-efficient Adversarial Imitation Learning from Observation
  3. 2023Adversarial Imitation Learning from Visual Observations using Latent Information
  4. 2023Sample-efficient Adversarial Imitation Learning
  5. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
  6. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees
  7. 2020State-only Imitation with Transition Dynamics Mismatch
  8. 2021Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency