mlsanityData to 7 Oct 2026

Matching Multiple Experts: On the Exploitability of Multi-Agent Imitation Learning

arXiv 2602.21020 ↗First posted 24 Feb 2026cs.LG cs.GT cs.MAcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Multi-agent imitation learning (MA-IL) aims to learn optimal policies from expert demonstrations of interactions in multi-agent interactive domains. Despite existing guarantees on the performance of the resulting learned policies, characterizations of how far the learned polices are from a Nash equilibrium are missing for offline MA-IL. In this paper, we demonstrate impossibility and hardness results of learning low-exploitable policies in general n-player Markov Games. We do so by providing examples where even exact measure matching fails, and demonstrating a new hardness result on characterizing the Nash gap given a fixed measure matching error. We then show how these challenges can be overcome using strategic dominance assumptions on the expert equilibrium. Specifically, for the case of dominant strategy expert equilibria, assuming Behavioral Cloning error εBC, this provides a Nash imitation gap of Oleft(nεBC/(1-γ)2right) for a discount factor γ. We generalize this result with a new notion of best-response continuity, and argue that this is implicitly encouraged by standard regularization techniques.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 32 of its 54 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

32 of 54 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learning23 May
  2. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learning17 Oct
  3. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning20 Jul
  4. 2024Multi-Agent Imitation Learning: Value is Easy, Regret is Hard6 Jun
  5. 2022Conditional Imitation Learning for Multi-Agent Games5 Jan
  6. 2021Multi-Agent Imitation Learning with Copulas10 Jul
  7. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  8. 2018Multi-Agent Generative Adversarial Imitation Learning26 Jul

Other references · 24

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  4. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  5. 2019A Theory of Regularized Markov Decision Processescited by 279
  6. 2023Imitating Human Behaviour with Diffusion Modelscited by 260
  7. 2018Understanding the impact of entropy on policy optimizationcited by 171
  8. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  9. 2022On the Convergence Rates of Policy Gradient Methodscited by 115
  10. 2020The Complexity of Constrained Min-Max Optimizationcited by 81
  11. 2017Online Reinforcement Learning in Stochastic Gamescited by 67
  12. 2018Multi-Agent Imitation Learning for Driving Simulationcited by 49
12 more
  1. 2023Uncoupled and Convergent Learning in Two-Player Zero-Sum Markov Games with Bandit Feedbackcited by 28
  2. 2022When is Offline Two-Player Zero-Sum Markov Game Solvable?cited by 27
  3. 2021Maximum-Entropy Multi-Agent Dynamic Games: Forward and Inverse Solutionscited by 22
  4. 2025Leveraging Large Language Models for Effective and Explainable Multi-Agent Credit Assignmentcited by 17
  5. 2023On Imitation in Mean-field Gamescited by 10
  6. 2024Multi-Agent Inverse Reinforcement Learning in Real World Unstructured Pedestrian Crowdscited by 6
  7. 2024Independent Policy Mirror Descent for Markov Potential Games: Scaling to Large Number of Playerscited by 6
  8. 2018Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Gamescited by 4
  9. 2024On Feasible Rewards in Multi-Agent Inverse Reinforcement Learningcited by 4
  10. 2024Tractable Equilibrium Computation in Markov Games through Risk Aversioncited by 3
  11. 2023Active Inverse Learning in Stackelberg Trajectory Gamescited by 1
  12. 2025CRAFT: Coaching Reinforcement Learning Autonomously using Foundation Models for Multi-Robot Coordination Taskscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022A Ranking Game for Imitation Learning
  2. 2023Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Competitive Games
  3. 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap
  4. 2026Multi-agent imitation learning with function approximation: Linear Markov games and beyond
  5. 2026Population-Aware Imitation Learning in Mean-field Games with Common Noise
  6. 2019Reinforced Imitation in Heterogeneous Action Space
  7. 2025Imitation Learning as Return Distribution Matching
  8. 2023Inverse Factorized Q-Learning for Cooperative Multi-agent Imitation Learning