# Imitation with Neural Density Models

arXiv: https://arxiv.org/abs/2010.09808 · HTML: https://mlsanity.com/paper/2010.09808/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 19 Oct 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Built on by (most cited first, 8 of 8)

- [Goal-Conditioned Imitation Learning using Score-based Diffusion Policies](https://mlsanity.com/paper/2304.02532/) (2023)
- [SEABO: A Simple Search-Based Method for Offline Imitation Learning](https://mlsanity.com/paper/2402.03807/) (2024)
- [IL-flOw: Imitation Learning from Observation using Normalizing Flows](https://mlsanity.com/paper/2205.09251/) (2022)
- [A Coupled Flow Approach to Imitation Learning](https://mlsanity.com/paper/2305.00303/) (2023)
- [Imitation Learning by State-Only Distribution Matching](https://mlsanity.com/paper/2202.04332/) (2022)
- [Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learning](https://mlsanity.com/paper/2402.01057/) (2024)
- [Occupancy Information Ratio: Infinite-Horizon, Information-Directed, Parameterized Policy Search](https://mlsanity.com/paper/2201.08832/) (2022)
- [Score-Based Training for Energy-Based TTS Models](https://mlsanity.com/paper/2505.13771/) (2025)

## Abstract (arXiv)

We propose a new framework for Imitation Learning (IL) via density estimation of the expert's occupancy measure followed by Maximum Occupancy Entropy Reinforcement Learning (RL) using the density as a reward. Our approach maximizes a non-adversarial model-free RL objective that provably lower bounds reverse Kullback-Leibler divergence between occupancy measures of the expert and imitator. We present a practical IL algorithm, Neural Density Imitation (NDI), which obtains state-of-the-art demonstration efficiency on benchmark control tasks.