mlsanityData to 7 Oct 2026

Regularizing Adversarial Imitation Learning Using Causal Invariance

arXiv 2308.09189 ↗First posted 17 Aug 2023cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning methods are used to infer a policy in a Markov decision process from a dataset of expert demonstrations by minimizing a divergence measure between the empirical state occupancy measures of the expert and the policy. The guiding signal to the policy is provided by the discriminator used as part of an versarial optimization procedure. We observe that this model is prone to absorbing spurious correlations present in the expert data. To alleviate this issue, we propose to use causal invariance as a regularization principle for adversarial training of these models. The regularization objective is applicable in a straightforward manner to existing adversarial imitation frameworks. We demonstrate the efficacy of the regularized formulation in an illustrative two-dimensional setting as well as a number of high-dimensional robot locomotion benchmark tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 20 of its 24 references that are in the map. First posted on arXiv 17 Aug 2023.

Builds on

20 of 24 references.

Closest ancestors · 4

Same direction, newest first

  1. 2019Task-Relevant Adversarial Imitation Learning2 Oct
  2. 2019Causal Confusion in Imitation Learning28 May
  3. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct
  4. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep

Other references · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Improved Training of Wasserstein GANscited by 2,223
  3. 2019Invariant Risk Minimizationcited by 1,832
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  6. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  7. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  8. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  9. 2020Invariant Risk Minimization Gamescited by 188
  10. 2022Defining and Characterizing Reward Hackingcited by 184
  11. 2017Invariant Causal Prediction for Nonlinear Modelscited by 140
  12. 2020Invariant Rationalizationcited by 131
4 more
  1. 2020Invariant Causal Prediction for Block MDPscited by 104
  2. 2020On Mixup Regularizationcited by 82
  3. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
  4. 2020Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learningcited by 37

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
  2. 2022Model-Based Imitation Learning Using Entropy Regularization of Model and Policy
  3. 2020ADAIL: Adaptive Adversarial Imitation Learning
  4. 2023Adversarial Imitation Learning from Visual Observations using Latent Information
  5. 2019Wasserstein Adversarial Imitation Learning
  6. 2022Latent Policies for Adversarial Imitation Learning
  7. 2023Invariant Causal Imitation Learning for Generalizable Policies
  8. 2023Sample-efficient Adversarial Imitation Learning