mlsanityData to 7 Oct 2026

Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systems

arXiv 1909.11468 ↗First posted 25 Sep 2019cs.MAcited by 7

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Many tasks in practice require the collaboration of multiple agents through reinforcement learning. In general, cooperative multiagent reinforcement learning algorithms can be classified into two paradigms: Joint Action Learners (JALs) and Independent Learners (ILs). In many practical applications, agents are unable to observe other agents' actions and rewards, making JALs inapplicable. In this work, we focus on independent learning paradigm in which each agent makes decisions based on its local observations only. However, learning is challenging in independent settings due to the local viewpoints of all agents, which perceive the world as a non-stationary environment due to the concurrently exploring teammates. In this paper, we propose a novel framework called Independent Generative Adversarial Self-Imitation Learning (IGASIL) to address the coordination problems in fully cooperative multiagent environments. To the best of our knowledge, we are the first to combine self-imitation learning with generative adversarial imitation learning (GAIL) and apply it to cooperative multiagent systems. Besides, we put forward a Sub-Curriculum Experience Replay mechanism to pick out the past beneficial experiences as much as possible and accelerate the self-imitation learning process. Evaluations conducted in the testbed of StarCraft unit micromanagement and a commonly adopted benchmark show that our IGASIL produces state-of-the-art results and even outperforms JALs in terms of both convergence speed and final performance.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 22 of its 50 references that are in the map. First posted on arXiv 25 Sep 2019.

Builds on

22 of 50 references.

Closest ancestors · 3

Same direction, newest first

  1. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  2. 2018Multi-Agent Generative Adversarial Imitation Learning26 Jul
  3. 2018Self-Imitation Learning14 Jun

Other references · 19

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  5. 2016End to End Learning for Self-Driving Carscited by 1,000
  6. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  7. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  8. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  9. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  10. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
  11. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  12. 2017Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observabilitycited by 143
7 more
  1. 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 114
  2. 2017Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Gamescited by 93
  3. 2018Multi-Agent Imitation Learning for Driving Simulationcited by 49
  4. 2017Guided Deep Reinforcement Learning for Swarm Systemscited by 44
  5. 2017Lenient Multi-Agent Deep Reinforcement Learningcited by 35
  6. 2018Learning from Demonstration in the Wildcited by 25
  7. 2018Towards Cooperation in Sequential Prisoner's Dilemmas: a Deep Multiagent Reinforcement Learning Approachcited by 4

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 2; 2024: 1; 2025: 1.

7 papers.

  1. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  2. 2022ERL-Re2: Efficient Evolutionary Reinforcement Learning with Shared State Representation and Individual Policy Representationcited by 11
  3. 2021Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problemcited by 5
  4. 2022DM2: Decentralized Multi-Agent Reinforcement Learning for Distribution Matchingcited by 5
  5. 2023AIIR-MIX: Multi-Agent Reinforcement Learning Meets Attention Individual Intrinsic Reward Mixing Networkcited by 1
  6. 2023DIRECT: Learning from Sparse and Shifting Rewards using Discriminative Reward Co-Trainingcited by 0
  7. 2025Autonomous Learning From Success and Failure: Goal-Conditioned Supervised Learning with Negative Feedbackcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Solo to Symphony: Orchestrating Multi-Agent Collaboration with Single-Agent Demos
  2. 2024Learning to Imitate Spatial Organization in Multi-robot Systems
  3. 2026Coordinated Diffusion: Generating Multi-Agent Behavior Without Multi-Agent Demonstrations
  4. 2018Generative Adversarial Self-Imitation Learning
  5. 2021SS-MAIL: Self-Supervised Multi-Agent Imitation Learning
  6. 2026Generative adversarial imitation learning for robot swarms: Learning from human demonstrations and trained policies
  7. 2024Multi-Agent Generative Adversarial Interactive Self-Imitation Learning for AUV Formation Control and Obstacle Avoidance
  8. 2020Multi-Agent Interactions Modeling with Correlated Policies