# IL-flOw: Imitation Learning from Observation using Normalizing Flows

arXiv: https://arxiv.org/abs/2205.09251 · HTML: https://mlsanity.com/paper/2205.09251/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 14 of its 19 references that are in the map.
- First posted on arXiv 19 May 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 14 of 14)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Why Normalizing Flows Fail to Detect Out-of-Distribution Data](https://mlsanity.com/paper/2006.08545/) (2020)
- [A Divergence Minimization Perspective on Imitation Learning Methods](https://mlsanity.com/paper/1911.02256/) (2019)
- [Primal Wasserstein Imitation Learning](https://mlsanity.com/paper/2006.04678/) (2020)
- [SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds](https://mlsanity.com/paper/2006.04604/) (2020)
- [f-IRL: Inverse Reinforcement Learning via State Marginal Matching](https://mlsanity.com/paper/2011.04709/) (2020)
- [Noise Regularization for Conditional Density Estimation](https://mlsanity.com/paper/1907.08982/) (2019)
- [Imitation with Neural Density Models](https://mlsanity.com/paper/2010.09808/) (2020)

## Built on by (most cited first, 12 of 12)

- [FlowPolicy: Enabling Fast and Robust 3D Flow-based Policy via Consistency Flow Matching for Robot Manipulation](https://mlsanity.com/paper/2412.04987/) (2024)
- [AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policies](https://mlsanity.com/paper/2402.04292/) (2024)
- [Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directions](https://mlsanity.com/paper/2402.13777/) (2024)
- [A Coupled Flow Approach to Imitation Learning](https://mlsanity.com/paper/2305.00303/) (2023)
- [TeLoGraF: Temporal Logic Planning via Graph-encoded Flow Matching](https://mlsanity.com/paper/2505.00562/) (2025)
- [Imitation Learning from Observation through Optimal Transport](https://mlsanity.com/paper/2310.01632/) (2023)
- [State Advantage Weighting for Offline RL](https://mlsanity.com/paper/2210.04251/) (2022)
- [CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/2210.09496/) (2022)
- [Generalizable Imitation Learning Through Pre-Trained Representations](https://mlsanity.com/paper/2311.09350/) (2023)
- [Imitation from Observation With Bootstrapped Contrastive Learning](https://mlsanity.com/paper/2302.06540/) (2023)
- [Composite Gaussian Processes Flows for Learning Discontinuous Multimodal Policies](https://mlsanity.com/paper/2502.01913/) (2025)
- [Domain-Invariant Per-Frame Feature Extraction for Cross-Domain Imitation Learning with Visual Observations](https://mlsanity.com/paper/2502.02867/) (2025)

## Abstract (arXiv)

We present an algorithm for Inverse Reinforcement Learning (IRL) from expert state observations only. Our approach decouples reward modelling from policy learning, unlike state-of-the-art adversarial methods which require updating the reward model during policy search and are known to be unstable and difficult to optimize. Our method, IL-flOw, recovers the expert policy by modelling state-state transitions, by generating rewards using deep density estimators trained on the demonstration trajectories, avoiding the instability issues of adversarial methods. We demonstrate that using the state transition log-probability density as a reward signal for forward reinforcement learning translates to matching the trajectory distribution of the expert demonstrations, and experimentally show good recovery of the true reward signal as well as state of the art results for imitation from observation on locomotion and robotic continuous control tasks.