mlsanityData to 7 Oct 2026

Combating False Negatives in Adversarial Imitation Learning

arXiv 2002.00412 ↗First posted 2 Feb 2020cs.LG cs.AI stat.MLcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

In adversarial imitation learning, a discriminator is trained to differentiate agent episodes from expert demonstrations representing the desired behavior. However, as the trained policy learns to be more successful, the negative examples (the ones produced by the agent) become increasingly similar to expert ones. Despite the fact that the task is successfully accomplished in some of the agent's trajectories, the discriminator is trained to output low values for them. We hypothesize that this inconsistent training signal for the discriminator can impede its learning, and consequently leads to worse overall performance of the agent. We show experimental evidence for this hypothesis and that the 'False Negatives' (i.e. successful agent episodes) significantly hinder adversarial imitation learning, which is the first contribution of this paper. Then, we propose a method to alleviate the impact of false negatives and test it on the BabyAI environment. This method consistently improves sample efficiency over the baselines by at least an order of magnitude.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 12 of its 22 references that are in the map. First posted on arXiv 2 Feb 2020.

Builds on

12 of 22 references.

Closest ancestors · 2

Same direction, newest first

  1. 2019Task-Relevant Adversarial Imitation Learning2 Oct
  2. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep

Other references · 10

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  3. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  4. 2018Learning to Navigate in Cities Without a Mapcited by 103
  5. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  6. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  7. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  8. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  9. 2015Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequencescited by 50
  10. 2019Positive-Unlabeled Reward Learningcited by 36

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 0; 2022: 1; 2023: 1; 2024: 0; 2025: 1.

4 papers.

  1. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
  2. 2020Offline Learning from Demonstrations and Unlabeled Experiencecited by 72
  3. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrationscited by 7
  4. 2025SafeMIL: Learning Offline Safe Imitation Policy from Non-Preferred Trajectoriescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Unlabeled Imperfect Demonstrations in Adversarial Imitation Learning
  2. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization
  3. 2023Sample-efficient Adversarial Imitation Learning
  4. 2021What Matters for Adversarial Imitation Learning?
  5. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
  6. 2026Latent Wasserstein Adversarial Imitation Learning
  7. 2025Learning What to Do and What Not To Do: Offline Imitation from Expert and Undesirable Demonstrations
  8. 2023Adversarial Imitation Learning from Visual Observations using Latent Information