mlsanityData to 14 Oct 2026

Generative Adversarial Self-Imitation Learning

arXiv 1812.00950 ↗First posted 3 Dec 2018cs.LG cs.AI stat.MLcited by 25

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

This paper explores a simple regularizer for reinforcement learning by proposing Generative Adversarial Self-Imitation Learning (GASIL), which encourages the agent to imitate past good trajectories via generative adversarial imitation learning framework. Instead of directly maximizing rewards, GASIL focuses on reproducing past good trajectories, which can potentially make long-term credit assignment easier when rewards are sparse and delayed. GASIL can be easily combined with any policy gradient objective by using GASIL as a learned shaped reward function. Our experimental results show that GASIL improves the performance of proximal policy optimization on 2D Point Mass and MuJoCo environments with delayed reward and stochastic dynamics.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 29 references that are in the map. First posted on arXiv 3 Dec 2018.

Builds on

17 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2018Self-Imitation Learning14 Jun
  2. 2018Learning Self-Imitating Diverse Policies25 May

Other references · 15

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2016Generative Adversarial Text to Image Synthesiscited by 735
  6. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  7. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  8. 2017Neural Episodic Controlcited by 151
  9. 2016Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervisioncited by 103
  10. 2016Model-Free Episodic Controlcited by 102
  11. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 102
  12. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
3 more
  1. 2018Neural Program Synthesis with Priority Queue Trainingcited by 24
  2. 2018Recall Traces: Backtracking Models for Efficient Reinforcement Learningcited by 23
  3. 2016Deep Learning for Reward Design to Improve Monte Carlo Tree Search in ATARI Gamescited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 2; 2020: 1; 2021: 7; 2022: 3; 2023: 4; 2024: 6; 2025: 1; 2026: 1.

25 papers.

Sort:

By area:

  1. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 66
  2. 2021Self-Supervised Online Reward Shaping in Sparse-Reward Environmentscited by 37
  3. 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
  4. 2021Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environmentscited by 28
  5. 2021A Max-Min Entropy Framework for Reinforcement Learningcited by 28
  6. 2021Off-Policy Reinforcement Learning with Delayed Rewardscited by 25
  7. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  8. 2022Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrationscited by 15
  9. 2019Randomized Adversarial Imitation Learning for Autonomous Drivingcited by 6
  10. 2024Harnessing Network Effect for Fake News Mitigation: Selecting Debunkers via Self-Imitation Learningcited by 5
  11. 2019Hindsight Generative Adversarial Imitation Learningcited by 3
  12. 2020Adversarial Imitation Learning via Random Searchcited by 3
13 more
  1. 2024Multi-Agent Generative Adversarial Interactive Self-Imitation Learning for AUV Formation Control and Obstacle Avoidancecited by 3
  2. 2024Learning Diverse Policies with Soft Self-Generated Guidancecited by 3
  3. 2025Goal-based Self-Adaptive Generative Adversarial Imitation Learning (Goal-SAGAIL) for Multi-goal Robotic Manipulation Taskscited by 2
  4. 2021Parallelized and Randomized Adversarial Imitation Learning for Safety-Critical Self-Driving Vehiclescited by 1
  5. 2022Accelerating Self-Imitation Learning from Demonstrations via Policy Constraints and Q-Ensemblecited by 1
  6. 2023Boosting Reinforcement Learning and Planning with Demonstrations: A Surveycited by 1
  7. 2024Trajectory-Oriented Policy Optimization with Sparse Rewardscited by 1
  8. 2024Improving Dialogue Agents by Decomposing One Global Explicit Annotation with Local Implicit Multimodal Feedbackcited by 1
  9. 2023DIRECT: Learning from Sparse and Shifting Rewards using Discriminative Reward Co-Trainingcited by 0
  10. 2023Enhanced Generalization through Prioritization and Diversity in Self-Imitation Reinforcement Learning over Procedural Environments with Sparse Rewardscited by 0
  11. 2023Policy Optimization with Smooth Guidance Learned from State-Only Demonstrationscited by 0
  12. 2024Preference-Guided Reinforcement Learning for Efficient Explorationcited by 0
  13. 2026Match or Replay: Self Imitating Proximal Policy Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020On Computation and Generalization of Generative Adversarial Imitation Learning
  2. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
  3. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation
  4. 2019Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systems
  5. 2020Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Models
  6. 2021GAN-Based Interactive Reinforcement Learning from Demonstration and Human Evaluative Feedback
  7. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning
  8. 2019A New Framework for Query Efficient Active Imitation Learning