mlsanityData to 14 Oct 2026

On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning

arXiv 2411.15046 ↗First posted 22 Nov 2024cs.LGcited by 4

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Multi-agent Inverse Reinforcement Learning (MAIRL) aims to recover agent reward functions from expert demonstrations. We characterize the feasible reward set in Markov games, identifying all reward functions that rationalize a given equilibrium. However, equilibrium-based observations are often ambiguous: a single Nash equilibrium can correspond to many reward structures, potentially changing the game's nature in multi-agent systems. We address this by introducing entropy-regularized Markov games, which yield a unique equilibrium while preserving strategic incentives. For this setting, we provide a sample complexity analysis detailing how errors affect learned policy performance. Our work establishes theoretical foundations and practical insights for MAIRL.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 25 of its 52 references that are in the map. First posted on arXiv 22 Nov 2024.

Builds on

25 of 52 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024How does Inverse RL Scale to Large State Spaces? A Provably Efficient Approach6 Jun
  2. 2024Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms23 Feb
  3. 2023On Imitation in Mean-field Games26 Jun
  4. 2023Towards Theoretical Understanding of Inverse Reinforcement Learning25 Apr
  5. 2022Identifiability and generalizability from multiple experts in Inverse Reinforcement Learning22 Sep
  6. 2022Active Exploration for Inverse Reinforcement Learning18 Jul
  7. 2021Identifiability in inverse reinforcement learning7 Jun
  8. 2019Multi-Agent Adversarial Inverse Reinforcement Learning30 Jul
  9. 2018Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games26 Jun

Other references · 16

  1. 2016Concrete Problems in AI Safetycited by 1,642
  2. 2019The StarCraft Multi-Agent Challengecited by 597
  3. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 313
  4. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 311
  5. 2017Inverse Reward Designcited by 249
  6. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  7. 2018Multi-Agent Generative Adversarial Imitation Learningcited by 100
  8. 2020Model-Based Multi-Agent RL in Zero-Sum Markov Games with Near-Optimal Sample Complexitycited by 92
  9. 2020Adaptive Reward-Free Explorationcited by 77
  10. 2018Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertisingcited by 65
  11. 2022Pessimistic Minimax Value Iteration: Provably Efficient Equilibrium Learning from Offline Datasetscited by 44
  12. 2019Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learningcited by 33
4 more
  1. 2021Scaling up Mean Field Games with Online Mirror Descentcited by 32
  2. 2021Exploration-Exploitation in Multi-Agent Competition: Convergence with Bounded Rationalitycited by 21
  3. 2024Multi-Agent Imitation Learning: Value is Easy, Regret is Hardcited by 11
  4. 2022Inverse Game Theory for Stackelberg Games: the Blessing of Bounded Rationalitycited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Improving Zero-Shot Offline RL via Behavioral Task Samplingcited by 1
  2. 2026DIML: Differentiable Inverse Mechanism Learning from Behaviors of Multi-Agent Learning Trajectoriescited by 0
  3. 2026Matching Multiple Experts: On the Exploitability of Multi-Agent Imitation Learningcited by 0
  4. 2026Inverse Reinforcement Learning without an Optimal Demonstrator: A Feasible Reward Set Approachcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Multi-Agent Inverse Reinforcement Learning: Suboptimal Demonstrations and Alternative Solution Concepts
  2. 2021Task-Guided Inverse Reinforcement Learning Under Partial Information
  3. 2026Decoding Rewards in Competitive Games: Inverse Game Theory with Entropy Regularization
  4. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  5. 2022Weighted Maximum Entropy Inverse Reinforcement Learning
  6. 2025Statistical analysis of Inverse Entropy-regularized Reinforcement Learning
  7. 2019Inverse Dynamic Games Based on Maximum Entropy Inverse Reinforcement Learning
  8. 2025Inverse Reinforcement Learning via Convex Optimization