# Reparameterized Variational Divergence Minimization for Stable Imitation

arXiv: https://arxiv.org/abs/2006.10810 · HTML: https://mlsanity.com/paper/2006.10810/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 18 Jun 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Built on by (most cited first, 2 of 2)

- [f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2010.01207/) (2020)
- [Imitation Learning from Observations under Transition Model Disparity](https://mlsanity.com/paper/2204.11446/) (2022)

## Abstract (arXiv)

While recent state-of-the-art results for adversarial imitation-learning algorithms are encouraging, recent works exploring the imitation learning from observation (ILO) setting, where trajectories textitonly contain expert observations, have not been met with the same success. Inspired by recent investigations of f-divergence manipulation for the standard imitation learning setting(Ke et al., 2019; Ghasemipour et al., 2019), we here examine the extent to which variations in the choice of probabilistic divergence may yield more performant ILO algorithms. We unfortunately find that f-divergence minimization through reinforcement learning is susceptible to numerical instabilities. We contribute a reparameterization trick for adversarial imitation learning to alleviate the optimization challenges of the promising f-divergence minimization framework. Empirically, we demonstrate that our design choices allow for ILO algorithms that outperform baseline approaches and more closely match expert performance in low-dimensional continuous-control tasks.