Imitation with Neural Density Models
On the map: Offline RL and imitation learning from data → Imitation learning methods
Abstract (arXiv)
We propose a new framework for Imitation Learning (IL) via density estimation of the expert's occupancy measure followed by Maximum Occupancy Entropy Reinforcement Learning (RL) using the density as a reward. Our approach maximizes a non-adversarial model-free RL objective that provably lower bounds reverse Kullback-Leibler divergence between occupancy measures of the expert and imitator. We present a practical IL algorithm, Neural Density Imitation (NDI), which obtains state-of-the-art demonstration efficiency on benchmark control tasks.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 19 Oct 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2022: 3; 2023: 2; 2024: 2; 2025: 1.
8 papers.
- 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policiescited by 229
- 2024SEABO: A Simple Search-Based Method for Offline Imitation Learningcited by 14
- 2022IL-flOw: Imitation Learning from Observation using Normalizing Flowscited by 12
- 2023A Coupled Flow Approach to Imitation Learningcited by 12
- 2022Imitation Learning by State-Only Distribution Matchingcited by 6
- 2024Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learningcited by 2
- 2022Occupancy Information Ratio: Infinite-Horizon, Information-Directed, Parameterized Policy Searchcited by 0
- 2025Score-Based Training for Energy-Based TTS Modelscited by 0
- 2025Score-Based Training for Energy-Based TTS Models19 May 2025
- 2024SEABO: A Simple Search-Based Method for Offline Imitation Learning6 Feb 2024
- 2024Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learning1 Feb 2024
- 2023A Coupled Flow Approach to Imitation Learning29 Apr 2023
- 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policies5 Apr 2023
- 2022IL-flOw: Imitation Learning from Observation using Normalizing Flows19 May 2022
- 2022Imitation Learning by State-Only Distribution Matching9 Feb 2022
- 2022Occupancy Information Ratio: Infinite-Horizon, Information-Directed, Parameterized Policy Search21 Jan 2022
Similar, not cited
Close in wording, no citation either way
- 2021Probability Density Estimation Based Imitation Learning
- 2020Universal Value Density Estimation for Imitation Learning and Goal-Conditioned Reinforcement Learning
- 2023Conditional Kernel Imitation Learning for Continuous State Environments
- 2021Imitation Learning by Reinforcement Learning
- 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
- 2020Energy-Based Imitation Learning
- 2021Imitation by Predicting Observations
- 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions