mlsanityData to 7 Oct 2026

Multi-Agent Adversarial Inverse Reinforcement Learning

arXiv 1907.13220 ↗First posted 30 Jul 2019cs.LG stat.MLcited by 60

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Reinforcement learning agents are prone to undesired behaviors due to reward mis-specification. Finding a set of reward functions to properly guide agent behaviors is particularly challenging in multi-agent scenarios. Inverse reinforcement learning provides a framework to automatically acquire suitable reward functions from expert demonstrations. Its extension to multi-agent settings, however, is difficult due to the more complex notions of rational behaviors. In this paper, we propose MA-AIRL, a new framework for multi-agent inverse reinforcement learning, which is effective and scalable for Markov games with high-dimensional state-action space and unknown dynamics. We derive our algorithm based on a new solution concept and maximum pseudolikelihood estimation within an adversarial reward learning framework. In the experiments, we demonstrate that MA-AIRL can recover reward functions that are highly correlated with ground truth ones, and significantly outperforms prior methods in terms of policy imitation.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 20 of its 60 references that are in the map. First posted on arXiv 30 Jul 2019.

Builds on

20 of 60 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games26 Jun

Other references · 19

  1. 2016Concrete Problems in AI Safetycited by 1,582
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  7. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  8. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  9. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  10. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  11. 2017Inverse Reward Designcited by 246
  12. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
7 more
  1. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  2. 2017Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximationcited by 119
  3. 2018Multi-Agent Generative Adversarial Imitation Learningcited by 99
  4. 2017Coordinated Multi-Agent Imitation Learningcited by 70
  5. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  6. 2013Computational Rationalization: The Inverse Equilibrium Problemcited by 24
  7. 2016Inverse Reinforcement Learning in Swarm Systemscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 10; 2022: 11; 2023: 7; 2024: 11; 2025: 13; 2026: 6.

60 papers.

Sort:

By area:

  1. 2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 43
  2. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  3. 2024Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directionscited by 20
  4. 2022A General Recipe for Likelihood-free Bayesian Optimizationcited by 19
  5. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matchingcited by 15
  6. 2021Multi-Agent Imitation Learning with Copulascited by 13
  7. 2022Conditional Imitation Learning for Multi-Agent Gamescited by 13
  8. 2023Multiagent Inverse Reinforcement Learning via Theory of Mind Reasoningcited by 11
  9. 2022Individual-Level Inverse Reinforcement Learning for Mean Field Gamescited by 10
  10. 2021Adversarial Inverse Reinforcement Learning for Mean Field Gamescited by 8
  11. 2021Learning Altruistic Behaviours in Reinforcement Learning without External Rewardscited by 8
  12. 2022On-the-fly Strategy Adaptation for ad-hoc Agent Coordinationcited by 7
38 more
  1. 2020Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Criticcited by 6
  2. 2021Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Drivingcited by 6
  3. 2021Learning Human Rewards by Inferring Their Latent Intelligence Levels in Multi-Agent Games: A Theory-of-Mind Approach with Application to Driving Datacited by 6
  4. 2024Multi-Agent Inverse Reinforcement Learning in Real World Unstructured Pedestrian Crowdscited by 6
  5. 2024Grounded Answers for Multi-agent Decision-making Problem through Generative World Modelcited by 6
  6. 2022Synthetic Data-Based Simulators for Recommender Systems: A Surveycited by 5
  7. 2023Curricular Subgoals for Inverse Reinforcement Learningcited by 5
  8. 2025DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learningcited by 5
  9. 2021Multi-Agent Advisor Q-Learningcited by 4
  10. 2021Robust Entropy-regularized Markov Decision Processescited by 4
  11. 2024Select to Perfect: Imitating desired behavior from large multi-agent datacited by 4
  12. 2024On Feasible Rewards in Multi-Agent Inverse Reinforcement Learningcited by 4
  13. 2025Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learningcited by 4
  14. 2021Equilibrium Inverse Reinforcement Learning for Ride-hailing Vehicle Networkcited by 3
  15. 2022The Multi-Agent Pickup and Delivery Problem: MAPF, MARL and Its Warehouse Applicationscited by 3
  16. 2022Identifying Coordination in a Cognitive Radar Network -- A Multi-Objective Inverse Reinforcement Learning Approachcited by 3
  17. 2023Utility Theory based Cognitive Modeling in the Application of Robotics: A Surveycited by 3
  18. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Reviewcited by 3
  19. 2025Efficient Inverse Multiagent Learningcited by 3
  20. 2025MisoDICE: Multi-Agent Imitation from Unlabeled Mixed-Quality Demonstrationscited by 3
  21. 2025Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variablescited by 3
  22. 2022Semi-Supervised Imitation Learning of Team Policies from Suboptimal Demonstrationscited by 2
  23. 2022Imitating Opponent to Win: Adversarial Policy Imitation Learning in Two-player Competitive Gamescited by 2
  24. 2023Discovering Individual Rewards in Collective Behavior through Inverse Multi-Agent Reinforcement Learningcited by 2
  25. 2023Inverse Factorized Q-Learning for Cooperative Multi-agent Imitation Learningcited by 2
  26. 2024Scalable Interactive Machine Learning for Future Command and Controlcited by 2
  27. 2024Cooperation and Control in Delegation Gamescited by 2
  28. 2024ARMCHAIR: integrated inverse reinforcement learning and model predictive control for human-robot collaborationcited by 2
  29. 2025Hierarchical Imitation Learning of Team Behavior from Heterogeneous Demonstrationscited by 2
  30. 2025Rate optimal learning of equilibria from datacited by 2
  31. 2023Global Nash Equilibrium in Non-convex Multi-player Game: Theory and Algorithmscited by 1
  32. 2025Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challengescited by 1
  33. 2025Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithmcited by 1
  34. 2026Multi-agent imitation learning with function approximation: Linear Markov games and beyondcited by 1
  35. 2022Learning to Advise and Learning from Advice in Cooperative Multi-Agent Reinforcement Learningcited by 0
  36. 2022Weighted Maximum Entropy Inverse Reinforcement Learningcited by 0
  37. 2023FP-IRL: Fokker--Planck Inverse Reinforcement Learning -- A Physics-Constrained Approach to Markov Decision Processescited by 0
  38. 2024Mean Field Correlated Imitation Learningcited by 0

50 most cited of 60 All 60 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shaping
  2. 2020oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actions
  3. 2024Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery and Algebraic Equilibrium Proof
  4. 2020Off-Policy Adversarial Inverse Reinforcement Learning
  5. 2018Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations
  6. 2021Meta-Adversarial Inverse Reinforcement Learning for Decision-making Tasks
  7. 2025Hybrid-AIRL: Enhancing Inverse Reinforcement Learning with Supervised Expert Guidance
  8. 2024On Reward Transferability in Adversarial Inverse Reinforcement Learning: Insights from Random Matrix Theory