mlsanityData to 7 Oct 2026

Task-Relevant Adversarial Imitation Learning

arXiv 1910.01077 ↗First posted 2 Oct 2019cs.LG cs.AI cs.ROcited by 45

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We show that a critical vulnerability in adversarial imitation is the tendency of discriminator networks to learn spurious associations between visual features and expert labels. When the discriminator focuses on task-irrelevant features, it does not provide an informative reward signal, leading to poor task performance. We analyze this problem in detail and propose a solution that outperforms standard Generative Adversarial Imitation Learning (GAIL). Our proposed method, Task-Relevant Adversarial Imitation Learning (TRAIL), uses constrained discriminator optimization to learn informative rewards. In comprehensive experiments, we show that TRAIL can solve challenging robotic manipulation tasks from pixels by imitating human operators without access to any task rewards, and clearly outperforms comparable baseline imitation agents, including those trained via behaviour cloning and conventional GAIL.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 58 references that are in the map. First posted on arXiv 2 Oct 2019.

Builds on

31 of 58 references.

Closest ancestors · 5

Same direction, newest first

  1. 2019Causal Confusion in Imitation Learning28 May
  2. 2019Reinforced Imitation in Heterogeneous Action Space6 Apr
  3. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct
  4. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  5. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets6 Sep

Other references · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  3. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  6. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  7. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  8. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  9. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  10. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 408
  11. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  12. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
14 more
  1. 2018Distributed Prioritized Experience Replaycited by 260
  2. 2017One-Shot Imitation Learningcited by 232
  3. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  4. 2018Distributed Distributional Deterministic Policy Gradientscited by 195
  5. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  6. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  7. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 115
  8. 2017Third-Person Imitation Learningcited by 103
  9. 2017Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstrationcited by 100
  10. 2017Learning human behaviors from motion capture by adversarial imitationcited by 75
  11. 2019Off-Policy Evaluation in Partially Observable Environmentscited by 65
  12. 2018Observe and Look Further: Achieving Consistent Performance on Ataricited by 42
  13. 2018Deconfounding Reinforcement Learning in Observational Settingscited by 36
  14. 2018Learning from Demonstration in the Wildcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 4; 2021: 10; 2022: 8; 2023: 11; 2024: 7; 2025: 2; 2026: 2.

45 papers.

Sort:

By area:

  1. 2022A Generalist Agentcited by 802
  2. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  3. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 76
  4. 2020Offline Learning from Demonstrations and Unlabeled Experiencecited by 72
  5. 2021Shaking the foundations: delusions in sequence models for interaction and controlcited by 61
  6. 2022Vision-Based Manipulators Need to Also See from Their Handscited by 46
  7. 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 44
  8. 2021Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classificationcited by 38
  9. 2023Diffusion Model-Augmented Behavioral Cloningcited by 37
  10. 2019Positive-Unlabeled Reward Learningcited by 36
  11. 2021Self-Supervised Disentangled Representation Learning for Third-Person Imitation Learningcited by 23
  12. 2020Semi-supervised reward learning for offline reinforcement learningcited by 22
33 more
  1. 2021Domain-Robust Visual Imitation Learning with Mutual Information Constraintscited by 20
  2. 2021Adversarial Training is Not Ready for Robot Learningcited by 20
  3. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  4. 2024Diffusion-Reward Adversarial Imitation Learningcited by 18
  5. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 18
  6. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learningcited by 15
  7. 2022An Empirical Investigation of Representation Learning for Imitationcited by 15
  8. 2024Solving Minimum-Cost Reach Avoid using Reinforcement Learningcited by 14
  9. 2021Imitation by Predicting Observationscited by 13
  10. 2023CEIL: Generalized Contextual Imitation Learningcited by 12
  11. 2022Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Taskscited by 10
  12. 2023SeMAIL: Eliminating Distractors in Visual Imitation via Separated Modelscited by 10
  13. 2023Inverse Reinforcement Learning with the Average Reward Criterioncited by 8
  14. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrationscited by 7
  15. 2023Offline Imitation Learning with Variational Counterfactual Reasoningcited by 7
  16. 2025Subtask-Aware Visual Reward Learning from Segmented Demonstrationscited by 7
  17. 2020Combating False Negatives in Adversarial Imitation Learningcited by 4
  18. 2021Adversarially Regularized Policy Learning Guided by Trajectory Optimizationcited by 4
  19. 2023Improving Generalization in Game Agents with Data Augmentation in Imitation Learningcited by 4
  20. 2021Back to Reality for Imitation Learningcited by 3
  21. 2022Latent Policies for Adversarial Imitation Learningcited by 3
  22. 2022Exploring the trade off between human driving imitation and safety for traffic simulationcited by 3
  23. 2023Contrastive Example-Based Controlcited by 3
  24. 2024Fusion Dynamical Systems with Machine Learning in Imitation Learning: A Comprehensive Overviewcited by 3
  25. 2024Learning Causally Invariant Reward Functions from Diverse Demonstrationscited by 2
  26. 2022Modeling Human Behavior Part I -- Learning and Belief Approachescited by 1
  27. 2024SENSOR: Imitate Third-Person Expert's Behaviors via Active Sensoringcited by 1
  28. 2026Semi-Supervised Cross-Domain Imitation Learningcited by 1
  29. 2021Learning Lipschitz Feedback Policies from Expert Demonstrations: Closed-Loop Guarantees, Generalization and Robustnesscited by 0
  30. 2023Specification-Guided Data Aggregation for Semantically Aware Imitation Learningcited by 0
  31. 2023Regularizing Adversarial Imitation Learning Using Causal Invariancecited by 0
  32. 2024Transductive Reward Inference on Graphcited by 0
  33. 2026Reinforcement Learning from Cross-domain Videos with Video Prediction Modelcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Sample-efficient Adversarial Imitation Learning
  2. 2023Unlabeled Imperfect Demonstrations in Adversarial Imitation Learning
  3. 2021RAIL: A modular framework for Reinforcement-learning-based Adversarial Imitation Learning
  4. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization
  5. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddings
  6. 2021Visual Adversarial Imitation Learning using Variational Models
  7. 2020Addressing reward bias in Adversarial Imitation Learning with neutral reward functions
  8. 2023Adversarial Imitation Learning from Visual Observations using Latent Information