mlsanityData to 14 Oct 2026

Multi-Agent Generative Adversarial Imitation Learning

arXiv 1807.09936 ↗First posted 26 Jul 2018cs.LG cs.AI cs.MAcited by 100

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning algorithms can be used to learn a policy from expert demonstrations without access to a reward signal. However, most existing approaches are not applicable in multi-agent settings due to the existence of multiple (Nash) equilibria and non-stationary environments. We propose a new framework for multi-agent imitation learning for general Markov games, where we build upon a generalized notion of inverse reinforcement learning. We further introduce a practical multi-agent actor-critic algorithm with good empirical performance. Our method can be used to imitate complex behaviors in high-dimensional environments with multiple cooperative or competing agents.

Key facts. Cited by 100 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 21 of its 59 references that are in the map. First posted on arXiv 26 Jul 2018.

Builds on

21 of 59 references.

References · 21

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  4. 2016Concrete Problems in AI Safetycited by 1,642
  5. 2016Generative Adversarial Imitation Learningcited by 1,597
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  7. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  8. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  9. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  10. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  11. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  12. 2016Cooperative Inverse Reinforcement Learningcited by 300
9 more
  1. 2017Inverse Reward Designcited by 249
  2. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 209
  3. 2018Constructing Unrestricted Adversarial Examples with Generative Modelscited by 135
  4. 2017Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximationcited by 121
  5. 2017Third-Person Imitation Learningcited by 103
  6. 2017Coordinated Multi-Agent Imitation Learningcited by 72
  7. 2016Inverse Reinforcement Learning in Swarm Systemscited by 18
  8. 2018Accelerating Natural Gradient with Higher-Order Invariancecited by 17
  9. 2015A Study of Gradient Descent Schemes for General-Sum Stochastic Gamescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 4; 2020: 4; 2021: 18; 2022: 14; 2023: 17; 2024: 19; 2025: 18; 2026: 5.

100 papers.

Sort:

By area:

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 148
  2. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gapcited by 81
  3. 2020Active Preference-Based Gaussian Process Regression for Reward Learningcited by 71
  4. 2019Multi-Agent Adversarial Inverse Reinforcement Learningcited by 60
  5. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 58
  6. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimizationcited by 49
  7. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 45
  8. 2024AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agentscited by 32
  9. 2021Towards Distraction-Robust Active Visual Trackingcited by 30
  10. 2021Deep Interactive Bayesian Reinforcement Learning via Meta-Learningcited by 29
  11. 2018Learning from Demonstration in the Wildcited by 25
  12. 2019Simulating Emergent Properties of Human Driving Behavior Using Multi-Agent Reward Augmented Imitation Learningcited by 24
38 more
  1. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  2. 2021Proximal Gradient Descent-Ascent: Variable Convergence under KL Geometrycited by 18
  3. 2021Provably Efficient Policy Optimization for Two-Player Zero-Sum Markov Gamescited by 18
  4. 2021TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrationscited by 18
  5. 2021Co-GAIL: Learning Diverse Strategies for Human-Robot Collaborationcited by 17
  6. 2024MAPF-GPT: Imitation Learning for Multi-Agent Pathfinding at Scalecited by 17
  7. 2023Diverse Conventions for Human-AI Collaborationcited by 15
  8. 2021Deep Exploration for Recommendation Systemscited by 14
  9. 2024Generative AI for Unmanned Vehicle Swarms: Challenges, Applications and Opportunitiescited by 14
  10. 2020Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Explorationcited by 13
  11. 2021Multi-Agent Imitation Learning with Copulascited by 13
  12. 2022Conditional Imitation Learning for Multi-Agent Gamescited by 13
  13. 2021Adversarial Imitation Learning with Trajectorial Augmentation and Correctioncited by 11
  14. 2024Multi-Agent Imitation Learning: Value is Easy, Regret is Hardcited by 11
  15. 2022Individual-Level Inverse Reinforcement Learning for Mean Field Gamescited by 10
  16. 2025From Screens to Scenes: A Survey of Embodied AI in Healthcarecited by 10
  17. 2023Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrationscited by 9
  18. 2021Adversarial Inverse Reinforcement Learning for Mean Field Gamescited by 8
  19. 2021Learning Altruistic Behaviours in Reinforcement Learning without External Rewardscited by 8
  20. 2023A first-order augmented Lagrangian method for constrained minimax optimizationcited by 8
  21. 2024LASIL: Learner-Aware Supervised Imitation Learning For Long-term Microscopic Traffic Simulationcited by 8
  22. 2019Independent Generative Adversarial Self-Imitation Learning in Cooperative Multiagent Systemscited by 7
  23. 2022RITA: Boost Driving Simulators with Realistic Interactive Traffic Flowcited by 7
  24. 2023Automated Task-Time Interventions to Improve Teamwork using Imitation Learningcited by 7
  25. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulatorcited by 7
  26. 2025MIMIC-D: Multi-modal Imitation for MultI-agent Coordination with Decentralized Diffusion Policiescited by 7
  27. 2019Sample-efficient Adversarial Imitation Learning from Observationcited by 6
  28. 2020Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Criticcited by 6
  29. 2021Learning to Simulate on Sparse Trajectory Datacited by 6
  30. 2023K-SHAP: Policy Clustering Algorithm for Anonymous Multi-Agent State-Action Pairscited by 6
  31. 2024Multi-Agent Inverse Reinforcement Learning in Real World Unstructured Pedestrian Crowdscited by 6
  32. 2024Grounded Answers for Multi-agent Decision-making Problem through Generative World Modelcited by 6
  33. 2025Direct Post-Training Preference Alignment for Multi-Agent Motion Generation Models Using Implicit Feedback from Pre-training Demonstrationscited by 6
  34. 2021A Cubic Regularization Approach for Finding Local Minimax Points in Nonconvex Minimax Optimizationcited by 5
  35. 2022DM2: Decentralized Multi-Agent Reinforcement Learning for Distribution Matchingcited by 5
  36. 2023iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learningcited by 5
  37. 2024Beyond Joint Demonstrations: Personalized Expert Guidance for Efficient Multi-Agent Reinforcement Learningcited by 5
  38. 2025Advancing Learnable Multi-Agent Pathfinding Solvers with Active Fine-Tuningcited by 5

50 most cited of 100 All 100 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Generating Personas for Games with Multimodal Adversarial Imitation Learning
  2. 2025Structured Imitation Learning of Interactive Policies through Inverse Games
  3. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
  4. 2025Enabling Off-Policy Imitation Learning with Deep Actor Critic Stabilization
  5. 2020Multi-Agent Interactions Modeling with Correlated Policies
  6. 2018Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations
  7. 2026Generative adversarial imitation learning for robot swarms: Learning from human demonstrations and trained policies
  8. 2021Imitation Learning by Reinforcement Learning