mlsanityData to 7 Oct 2026

Model-Based Imitation Learning Using Entropy Regularization of Model and Policy

arXiv 2206.10101 ↗First posted 21 Jun 2022cs.LG cs.AIcited by 1

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Approaches based on generative adversarial networks for imitation learning are promising because they are sample efficient in terms of expert demonstrations. However, training a generator requires many interactions with the actual environment because model-free reinforcement learning is adopted to update a policy. To improve the sample efficiency using model-based reinforcement learning, we propose model-based Entropy-Regularized Imitation Learning (MB-ERIL) under the entropy-regularized Markov decision process to reduce the number of interactions with the actual environment. MB-ERIL uses two discriminators. A policy discriminator distinguishes the actions generated by a robot from expert ones, and a model discriminator distinguishes the counterfactual state transitions generated by the model from the actual ones. We derive structured discriminators so that the learning of the policy and the model is efficient. Computer simulations and real robot experiments show that MB-ERIL achieves a competitive performance and significantly improves the sample efficiency compared to baseline methods.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 40 references that are in the map. First posted on arXiv 21 Jun 2022.

Builds on

23 of 40 references.

Closest ancestors · 11

Same direction, newest first

  1. 2021DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation31 Mar
  2. 2021Off-Policy Imitation Learning from Observations25 Feb
  3. 2020When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence24 Jun
  4. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization23 Jun
  5. 2020Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Rate8 Mar
  6. 2020Discriminator Soft Actor Critic without Extrinsic Rewards19 Jan
  7. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  8. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards27 May
  9. 2019Dyna-AIL : Adversarial Imitation Learning by Planning8 Mar
  10. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  11. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets6 Sep

Other references · 12

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2016OpenAI Gymcited by 1,684
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  7. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  8. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  9. 2019From Variational to Deterministic Autoencoderscited by 131
  10. 2016Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamicscited by 24
  11. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matchingcited by 15
  12. 2020Forward and inverse reinforcement learning sharing network weights and hyperparameterscited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Statistical analysis of Inverse Entropy-regularized Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
  2. 2023Regularizing Adversarial Imitation Learning Using Causal Invariance
  3. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model
  4. 2019Generative predecessor models for sample-efficient imitation learning
  5. 2025Generalization Capability for Imitation Learning
  6. 2020ADAIL: Adaptive Adversarial Imitation Learning
  7. 2022Latent Policies for Adversarial Imitation Learning
  8. 2024Efficient Imitation Learning with Conservative World Models