mlsanityData to 7 Oct 2026

Fail-Safe Adversarial Generative Imitation Learning

arXiv 2203.01696 ↗First posted 3 Mar 2022cs.LG cs.MA stat.MLcited by 2

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

For flexible yet safe imitation learning (IL), we propose theory and a modular method, with a safety layer that enables a closed-form probability density/gradient of the safe generative continuous policy, end-to-end generative adversarial training, and worst-case safety guarantees. The safety layer maps all actions into a set of safe actions, and uses the change-of-variables formula plus additivity of measures for the density. The set of safe actions is inferred by first checking safety of a finite sample of actions via adversarial reachability analysis of fallback maneuvers, and then concluding on the safety of these actions' neighborhoods using, e.g., Lipschitz continuity. We provide theoretical analysis showing the robustness advantage of using the safety layer already during training (imitation error linear in the horizon) compared to only using it at test time (up to quadratic error). In an experiment on real-world driver interaction data, we empirically demonstrate tractability, safety and imitation performance of our approach.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 56 of its 89 references that are in the map. First posted on arXiv 3 Mar 2022.

Builds on

56 of 89 references.

Closest ancestors · 11

Same direction, newest first

  1. 2021Policy Gradient Bayesian Robust Optimization for Imitation Learning11 Jun
  2. 2021On Imitation Learning of Linear Control Policies: Enforcing Stability and Robustness Constraints via LMI Conditions24 Mar
  3. 2021On the Sample Complexity of Stability Constrained Imitation Learning18 Feb
  4. 2020Imitation Learning with Stability and Safety Guarantees16 Dec
  5. 2020Error Bounds of Imitating Policies and Environments22 Oct
  6. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences21 Feb
  7. 2019Wasserstein Adversarial Imitation Learning19 Jun
  8. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  9. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  10. 2018Risk-Sensitive Generative Adversarial Imitation Learning13 Aug
  11. 2018Multi-Agent Generative Adversarial Imitation Learning26 Jul

Other references · 45

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  4. 2017Constrained Policy Optimizationcited by 928
  5. 2019Deep Equilibrium Modelscited by 603
  6. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  8. 2017Safe Reinforcement Learning via Shieldingcited by 427
  9. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  10. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  11. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 347
  12. 2018The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systemscited by 276
33 more
  1. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
  2. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
  3. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  4. 2018Safe Exploration in Continuous Action Spacescited by 237
  5. 2019Human Motion Trajectory Prediction: A Surveycited by 231
  6. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  7. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 186
  8. 2021DC3: A learning method for optimization with hard constraintscited by 181
  9. 2018Differentiable MPC for End-to-end Planning and Controlcited by 178
  10. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  11. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
  12. 2017On a Formal Model of Safe and Scalable Self-driving Carscited by 129
  13. 2019Implicit Deep Learningcited by 124
  14. 2020DSDNet: Deep Structured self-Driving Networkcited by 61
  15. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  16. 2019Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safetycited by 48
  17. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 43
  18. 2020Enforcing robust control guarantees within neural network policiescited by 40
  19. 2019Improving Exploration in Soft-Actor-Critic with Normalizing Flows Policiescited by 37
  20. 2021OVERT: An Algorithm for Safety Verification of Neural Network Control Policies for Nonlinear Systemscited by 37
  21. 2018Linear model predictive safety certification for learning-based controlcited by 34
  22. 2019Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learningcited by 24
  23. 2021Differentiable Trust Region Layers for Deep Reinforcement Learningcited by 24
  24. 2020Reinforcement Learning based Control of Imitative Policies for Near-Accident Drivingcited by 23
  25. 2018What game are we playing? End-to-end learning in normal and extensive form gamescited by 22
  26. 2019Learning a Decision Module by Imitating Driver's Control Behaviorscited by 17
  27. 2020Learning Game-Theoretic Models of Multiagent Trajectories Using Implicit Layerscited by 13
  28. 2020Spatiotemporal Learning of Multivehicle Interaction Patterns in Lane-Change Scenarioscited by 8
  29. 2020Double Oracle Algorithm for Computing Equilibria in Continuous Gamescited by 7
  30. 2019Large Scale Learning of Agent Rationality in Two-Player Zero-Sum Gamescited by 6
  31. 2016Computing Robust Controlled Invariant Sets of Linear Systemscited by 5
  32. 2020Approximate Solutions to a Class of Reachability Gamescited by 2
  33. 2021Learning soft interventions in complex equilibrium systemscited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 1.

2 papers.

  1. 2025Generative Models in Decision Making: A Surveycited by 14
  2. 2023On Learning the Tail Quantiles of Driving Behavior Distributions via Quantile Regression and Flowscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024SAFE-GIL: SAFEty Guided Imitation Learning for Robotic Systems
  2. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments
  3. 2026TAIL-Safe: Task-Agnostic Safety Monitoring for Imitation Learning Policies
  4. 2023Provably Efficient Adversarial Imitation Learning with Unknown Transitions
  5. 2025State-Conditional Adversarial Learning: An Off-Policy Visual Domain Transfer Method for End-to-End Imitation Learning
  6. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios
  7. 2021What Matters for Adversarial Imitation Learning?
  8. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets