mlsanityData to 14 Oct 2026

Probability Density Estimation Based Imitation Learning

arXiv 2112.06746 ↗First posted 13 Dec 2021cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation Learning (IL) is an effective learning paradigm exploiting the interactions between agents and environments. It does not require explicit reward signals and instead tries to recover desired policies using expert demonstrations. In general, IL methods can be categorized into Behavioral Cloning (BC) and Inverse Reinforcement Learning (IRL). In this work, a novel reward function based on probability density estimation is proposed for IRL, which can significantly reduce the complexity of existing IRL methods. Furthermore, we prove that the theoretically optimal policy derived from our reward function is identical to the expert policy as long as it is deterministic. Consequently, an IRL problem can be gracefully transformed into a probability density estimation problem. Based on the proposed reward function, we present a "watch-try-learn" style framework named Probability Density Estimation based Imitation Learning (PDEIL), which can work in both discrete and continuous action spaces. Finally, comprehensive experiments in the Gym environment show that PDEIL is much more efficient than existing algorithms in recovering rewards close to the ground truth.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 10 of its 33 references that are in the map. First posted on arXiv 13 Dec 2021.

Builds on

10 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2020Primal Wasserstein Imitation Learning8 Jun
  2. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Reward7 Jun
  3. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation16 May

Other references · 7

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  5. 2016Generative Adversarial Imitation Learningcited by 1,597
  6. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  7. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020Universal Value Density Estimation for Imitation Learning and Goal-Conditioned Reinforcement Learning
  2. 2023Conditional Kernel Imitation Learning for Continuous State Environments
  3. 2021Sample Efficient Imitation Learning via Reward Function Trained in Advance
  4. 2022A Ranking Game for Imitation Learning
  5. 2025Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement
  6. 2021Hyperparameter Selection for Imitation Learning
  7. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges
  8. 2020Imitation with Neural Density Models