mlsanityData to 7 Oct 2026

Fast Policy Learning through Imitation and Reinforcement

arXiv 1805.10413 ↗First posted 26 May 2018cs.LG stat.MLcited by 26

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning (IL) consists of a set of tools that leverage expert demonstrations to quickly learn policies. However, if the expert is suboptimal, IL can yield policies with inferior performance compared to reinforcement learning (RL). In this paper, we aim to provide an algorithm that combines the best aspects of RL and IL. We accomplish this by formulating several popular RL and IL algorithms in a common mirror descent framework, showing that these algorithms can be viewed as a variation on a single approach. We then propose LOKI, a strategy for policy learning that first performs a small but random number of IL iterations before switching to a policy gradient RL method. We show that if the switching time is properly randomized, LOKI can learn to outperform a suboptimal expert and converge faster than running policy gradient from scratch. Finally, we evaluate the performance of LOKI experimentally in several simulated environments.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 26 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 2; 2020: 1; 2021: 3; 2022: 6; 2023: 6; 2024: 3; 2025: 2; 2026: 1.

26 papers.

Sort:

By area:

  1. 2019Imitation Learning as f-Divergence Minimizationcited by 121
  2. 2019BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learningcited by 90
  3. 2023Learning to Generate Better Than Your LLMcited by 49
  4. 2023RLIF: Interactive Imitation Learning as Reinforcement Learningcited by 24
  5. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 24
  6. 2023Blending Imitation and Reinforcement Learning for Robust Policy Improvementcited by 16
  7. 2022Robots Enact Malignant Stereotypescited by 14
  8. 2022How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulationcited by 9
  9. 2022Combining imitation and deep reinforcement learning to accomplish human-level performance on a virtual foraging taskcited by 4
  10. 2022Self-Imitation Learning from Demonstrationscited by 4
  11. 2026Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Drivingcited by 4
  12. 2018Dynamic Regret Convergence Analysis and an Adaptive Regularization Algorithm for On-Policy Robot Imitation Learningcited by 3
14 more
  1. 2021Online Baum-Welch algorithm for Hierarchical Imitation Learningcited by 3
  2. 2021Bootstrapping Motor Skill Learning with Motion Planningcited by 2
  3. 2023Deploying Offline Reinforcement Learning with Human Feedbackcited by 2
  4. 2025Model-Based Diffusion Sampling for Predictive Control in Offline Decision Makingcited by 2
  5. 2020Explaining Fast Improvement in Online Imitation Learningcited by 1
  6. 2022Generalizing to New Tasks via One-Shot Compositional Subgoalscited by 1
  7. 2023Transferring Multiple Policies to Hotstart Reinforcement Learning in an Air Compressor Management Problemcited by 1
  8. 2023A Structured Prediction Approach for Robot Imitation Learningcited by 1
  9. 2018Learning Neural Parsers with Deterministic Differentiable Imitation Learningcited by 0
  10. 2021Leveraging Experience in Lazy Searchcited by 0
  11. 2022Neural-Progressive Hedging: Enforcing Constraints in Reinforcement Learning with Stochastic Programmingcited by 0
  12. 2024Accelerating Inverse Reinforcement Learning with Expert Bootstrappingcited by 0
  13. 2024A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observationscited by 0
  14. 2025TubeDAgger: Reducing the Number of Expert Interventions with Stochastic Reach-Tubescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018Accelerating Imitation Learning with Predictive Models
  2. 2019A Divergence Minimization Perspective on Imitation Learning Methods
  3. 2024Hybrid Inverse Reinforcement Learning
  4. 2019Reward-Conditioned Policies
  5. 2022Imitating, Fast and Slow: Robust learning from demonstrations via decision-time planning
  6. 2022Efficient Reinforcement Learning from Demonstration Using Local Ensemble and Reparameterization with Split and Merge of Expert Policies
  7. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions
  8. 2021Imitation Learning by Reinforcement Learning