mlsanityData to 14 Oct 2026

Error Bounds of Imitating Policies and Environments

arXiv 2010.11876 ↗First posted 22 Oct 2020cs.LGcited by 90

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning trains a policy by mimicking expert demonstrations. Various imitation methods were proposed and empirically evaluated, meanwhile, their theoretical understanding needs further studies. In this paper, we firstly analyze the value gap between the expert policy and imitated policies by two imitation methods, behavioral cloning and generative adversarial imitation. The results support that generative adversarial imitation can reduce the compounding errors compared to behavioral cloning, and thus has a better sample complexity. Noticed that by considering the environment transition model as a dual agent, imitation learning can also be used to learn the environment model. Therefore, based on the bounds of imitating policies, we further analyze the performance of imitating environments. The results show that environment models can be more effectively imitated by generative adversarial imitation than behavioral cloning, suggesting a novel application of adversarial imitation for model-based reinforcement learning. We hope these results could inspire future advances in imitation learning and model-based reinforcement learning.

Key facts. Cited by 90 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 22 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 19; 2023: 23; 2024: 24; 2025: 11; 2026: 8.

90 papers.

Sort:

By area:

  1. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 279
  2. 2026Self-Distillation Enables Continual Learningcited by 247
  3. 2024Preserving Diversity in Supervised Fine-Tuning of Large Language Modelscited by 70
  4. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 70
  5. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regressioncited by 66
  6. 2022A Survey on Model-based Reinforcement Learningcited by 66
  7. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learningcited by 63
  8. 2023What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?cited by 36
  9. 2025Generalist Reward Models: Found Inside Large Language Modelscited by 34
  10. 2022Active Exploration for Inverse Reinforcement Learningcited by 27
  11. 2022Proximal Point Imitation Learningcited by 26
  12. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learningcited by 26
38 more
  1. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruptioncited by 24
  2. 2023State Regularized Policy Optimization on Data with Dynamics Shiftcited by 23
  3. 2023Coherent Soft Imitation Learningcited by 22
  4. 2024Imitating Language via Scalable Inverse Reinforcement Learningcited by 21
  5. 2022Offline Reinforcement Learning with Causal Structured World Modelscited by 19
  6. 2024Vision-Language Navigation with Energy-Based Policycited by 19
  7. 2022A Ranking Game for Imitation Learningcited by 18
  8. 2023Plan To Predict: Learning an Uncertainty-Foreseeing Model for Model-Based Reinforcement Learningcited by 17
  9. 2024Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitationcited by 17
  10. 2022Delayed Reinforcement Learning by Imitationcited by 16
  11. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitionscited by 16
  12. 2022Rethinking ValueDice: Does It Really Improve Performance?cited by 15
  13. 2025Generative Models in Decision Making: A Surveycited by 15
  14. 2022Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysiscited by 13
  15. 2021Provably Breaking the Quadratic Error Compounding Barrier in Imitation Learning, Optimallycited by 12
  16. 2023Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learningcited by 12
  17. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximationcited by 11
  18. 2022Robust Imitation Learning from Corrupted Demonstrationscited by 10
  19. 2023On Imitation in Mean-field Gamescited by 10
  20. 2024Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Ratecited by 10
  21. 2024OASIS: Conditional Distribution Shaping for Offline Safe Reinforcement Learningcited by 10
  22. 2023A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learningcited by 9
  23. 2023Time-series Generation by Contrastive Imitationcited by 9
  24. 2024Learning from Sparse Offline Datasets via Conservative Density Estimationcited by 9
  25. 2024Active Fine-Tuning of Multi-Task Policiescited by 9
  26. 2023How To Guide Your Learner: Imitation Learning with Active Adaptive Expert Involvementcited by 8
  27. 2024Offline Imitation Learning with Model-based Reverse Augmentationcited by 8
  28. 2022RITA: Boost Driving Simulators with Realistic Interactive Traffic Flowcited by 7
  29. 2023A Survey on Offline Model-Based Reinforcement Learningcited by 7
  30. 2023Adversarial Constrained Bidding via Minimax Regret Optimization with Causality-Aware Reinforcement Learningcited by 7
  31. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learningcited by 7
  32. 2026On-Policy Distillation with Best-of-N Teacher Rollout Selectioncited by 7
  33. 2022Model-based Reinforcement Learning with Multi-step Plan Value Estimationcited by 6
  34. 2022Tight Performance Guarantees of Imitator Policies with Continuous Actionscited by 6
  35. 2024Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximationcited by 6
  36. 2022Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemblecited by 5
  37. 2022On Efficient Online Imitation Learning via Classificationcited by 5
  38. 2023Sim2Rec: A Simulator-based Decision-making Approach to Optimize Real-World Long-term User Engagement in Sequential Recommender Systemscited by 5

50 most cited of 90 All 90 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient Imitation Learning with Conservative World Models
  2. 2022Planning for Sample Efficient Imitation Learning
  3. 2022Robust Learning from Observation with Model Misspecification
  4. 2022Diverse Imitation Learning via Self-Organizing Generative Models
  5. 2025Enabling Off-Policy Imitation Learning with Deep Actor Critic Stabilization
  6. 2018Multi-Agent Generative Adversarial Imitation Learning
  7. 2020State-only Imitation with Transition Dynamics Mismatch
  8. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation