mlsanityData to 7 Oct 2026

Imitation Learning by Reinforcement Learning

arXiv 2108.04763 ↗First posted 10 Aug 2021stat.ML cs.LGcited by 17

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning algorithms learn a policy from demonstrations of expert behavior. We show that, for deterministic experts, imitation learning can be done by reduction to reinforcement learning with a stationary reward. Our theoretical analysis both certifies the recovery of expert reward and bounds the total variation distance between the expert and the imitation learner, showing a link to adversarial imitation learning. We conduct experiments which confirm that our reduction works well in practice for continuous control tasks.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 9 of its 20 references that are in the map. First posted on arXiv 10 Aug 2021.

Builds on

9 of 20 references.

Closest ancestors · 3

Same direction, newest first

  1. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  2. 2019Provably Efficient Imitation Learning from Observation Alone27 May
  3. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation16 May

Other references · 6

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2018Exploration by Random Network Distillationcited by 966
  4. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  5. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  6. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 5; 2025: 4; 2026: 1.

17 papers.

Sort:

By area:

  1. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
  2. 2023DITTO: Offline Imitation Learning with World Modelscited by 20
  3. 2024SEABO: A Simple Search-Based Method for Offline Imitation Learningcited by 14
  4. 2024Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learningcited by 10
  5. 2023A Strong Baseline for Batch Imitation Learningcited by 5
  6. 2025When Greedy Wins: Emergent Exploitation Bias in Meta-Bandit LLM Trainingcited by 3
  7. 2024Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learningcited by 2
  8. 2022Ask-AC: An Initiative Advisor-in-the-Loop Actor-Critic Frameworkcited by 1
  9. 2024Robot See, Robot Do: Imitation Reward for Noisy Financial Environmentscited by 1
  10. 2025Reinforcement Learning for Hanabicited by 1
  11. 2022Accelerated Continuous-Time Approximate Dynamic Programming via Data-Assisted Hybrid Controlcited by 0
  12. 2022Backward Curriculum Reinforcement Learningcited by 0
5 more
  1. 2023Imitator Learning: Achieve Out-of-the-Box Imitation Ability in Variable Environmentscited by 0
  2. 2024On the Complexity of Learning to Cooperate with Populations of Socially Rational Agentscited by 0
  3. 2025Imitation Learning of Correlated Policies in Stackelberg Gamescited by 0
  4. 2025Social Cooperation in Conversational AI Agentscited by 0
  5. 2026Imitation Learning for Elder-Facing Speech Synthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Reinforced Imitation in Heterogeneous Action Space
  2. 2019Reward-Conditioned Policies
  3. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards
  4. 2022Imitation Learning by Estimating Expertise of Demonstrators
  5. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning
  6. 2025The Pitfalls of Imitation Learning when Actions are Continuous
  7. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges
  8. 2019A New Framework for Query Efficient Active Imitation Learning