mlsanityData to 7 Oct 2026

IL-flOw: Imitation Learning from Observation using Normalizing Flows

arXiv 2205.09251 ↗First posted 19 May 2022cs.LG cs.AI cs.ROcited by 12

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We present an algorithm for Inverse Reinforcement Learning (IRL) from expert state observations only. Our approach decouples reward modelling from policy learning, unlike state-of-the-art adversarial methods which require updating the reward model during policy search and are known to be unstable and difficult to optimize. Our method, IL-flOw, recovers the expert policy by modelling state-state transitions, by generating rewards using deep density estimators trained on the demonstration trajectories, avoiding the instability issues of adversarial methods. We demonstrate that using the state transition log-probability density as a reward signal for forward reinforcement learning translates to matching the trajectory distribution of the expert demonstrations, and experimentally show good recovery of the true reward signal as well as state of the art results for imitation from observation on locomotion and robotic continuous control tasks.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 14 of its 19 references that are in the map. First posted on arXiv 19 May 2022.

Builds on

14 of 19 references.

Closest ancestors · 3

Same direction, newest first

  1. 2020Imitation with Neural Density Models19 Oct
  2. 2020Primal Wasserstein Imitation Learning8 Jun
  3. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov

Other references · 11

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2016Improved Techniques for Training GANscited by 2,938
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2019Neural Spline Flowscited by 590
  6. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  7. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  8. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 220
  9. 2020SoftFlow: Probabilistic Framework for Normalizing Flow on Manifoldscited by 97
  10. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
  11. 2019Noise Regularization for Conditional Density Estimationcited by 23

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 4; 2024: 3; 2025: 3.

12 papers.

  1. 2024FlowPolicy: Enabling Fast and Robust 3D Flow-based Policy via Consistency Flow Matching for Robot Manipulationcited by 99
  2. 2024AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policiescited by 50
  3. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  4. 2023A Coupled Flow Approach to Imitation Learningcited by 12
  5. 2025TeLoGraF: Temporal Logic Planning via Graph-encoded Flow Matchingcited by 9
  6. 2023Imitation Learning from Observation through Optimal Transportcited by 7
  7. 2022State Advantage Weighting for Offline RLcited by 5
  8. 2022CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrationscited by 4
  9. 2023Generalizable Imitation Learning Through Pre-Trained Representationscited by 3
  10. 2023Imitation from Observation With Bootstrapped Contrastive Learningcited by 1
  11. 2025Composite Gaussian Processes Flows for Learning Discontinuous Multimodal Policiescited by 0
  12. 2025Domain-Invariant Per-Frame Feature Extraction for Cross-Domain Imitation Learning with Visual Observationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
  2. 2024Inverse Delayed Reinforcement Learning
  3. 2022Imitation Learning from Observations under Transition Model Disparity
  4. 2022Imitation Learning by State-Only Distribution Matching
  5. 2022Robust Learning from Observation with Model Misspecification
  6. 2024Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment
  7. 2023Imitation Learning from Observation with Automatic Discount Scheduling
  8. 2024Inverse Reinforcement Learning by Estimating Expertise of Demonstrators