# Density Matching Reward Learning

arXiv: https://arxiv.org/abs/1608.03694 · HTML: https://mlsanity.com/paper/1608.03694/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2016.

## Built on by (most cited first, 4 of 4)

- [Strictly Batch Imitation Learning by Energy-based Distribution Matching](https://mlsanity.com/paper/2006.14154/) (2020)
- [Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations](https://mlsanity.com/paper/1801.02124/) (2018)
- [Safe Imitation Learning of Nonlinear Model Predictive Control for Flexible Robots](https://mlsanity.com/paper/2212.02941/) (2022)
- [SOCRATES: Text-based Human Search and Approach using a Robot Dog](https://mlsanity.com/paper/2302.05324/) (2023)

## Abstract (arXiv)

In this paper, we focus on the problem of inferring the underlying reward function of an expert given demonstrations, which is often referred to as inverse reinforcement learning (IRL). In particular, we propose a model-free density-based IRL algorithm, named density matching reward learning (DMRL), which does not require model dynamics. The performance of DMRL is analyzed theoretically and the sample complexity is derived. Furthermore, the proposed DMRL is extended to handle nonlinear IRL problems by assuming that the reward function is in the reproducing kernel Hilbert space (RKHS) and kernel DMRL (KDMRL) is proposed. The parameters for KDMRL can be computed analytically, which greatly reduces the computation time. The performance of KDMRL is extensively evaluated in two sets of experiments: grid world and track driving experiments. In grid world experiments, the proposed KDMRL method is compared with both model-based and model-free IRL methods and shows superior performance on a nonlinear reward setting and competitive performance on a linear reward setting in terms of expected value differences. Then we move on to more realistic experiments of learning different driving styles for autonomous navigation in complex and dynamic tracks using KDMRL and receding horizon control.