mlsanityData to 7 Oct 2026

Non-Cooperative Inverse Reinforcement Learning

arXiv 1911.04220 ↗First posted 3 Nov 2019cs.GT cs.AI cs.LGcited by 7

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Making decisions in the presence of a strategic opponent requires one to take into account the opponent's ability to actively mask its intended objective. To describe such strategic situations, we introduce the non-cooperative inverse reinforcement learning (N-CIRL) formalism. The N-CIRL formalism consists of two agents with completely misaligned objectives, where only one of the agents knows the true objective function. Formally, we model the N-CIRL formalism as a zero-sum Markov game with one-sided incomplete information. Through interacting with the more informed player, the less informed player attempts to both infer, and act according to, the true objective function. As a result of the one-sided incomplete information, the multi-stage game can be decomposed into a sequence of single-stage games expressed by a recursive formula. Solving this recursive formula yields the value of the N-CIRL game and the more informed player's equilibrium strategy. Another recursive formula, constructed by forming an auxiliary game, termed the dual game, yields the less informed player's strategy. Building upon these two recursive formulas, we develop a computationally tractable algorithm to approximately solve for the equilibrium strategies. Finally, we demonstrate the benefits of our N-CIRL formalism over the existing multi-agent IRL formalism via extensive numerical simulation in a novel cyber security setting.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 12 of its 51 references that are in the map. First posted on arXiv 3 Nov 2019.

Builds on

12 of 51 references.

Closest ancestors · 3

Same direction, newest first

  1. 2018Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games26 Jun
  2. 2018An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning11 Jun
  3. 2018Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations7 Jan

Other references · 9

  1. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  2. 2016Cooperative Inverse Reinforcement Learningcited by 298
  3. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
  4. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
  5. 2016Research Priorities for Robust and Beneficial Artificial Intelligencecited by 74
  6. 2019Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Gamescited by 38
  7. 2018Actor-Critic Policy Optimization in Partially Observable Multiagent Environmentscited by 35
  8. 2014Multi-agent Inverse Reinforcement Learning for Two-person Zero-sum Gamescited by 17
  9. 2018Finite-Sample Analysis For Decentralized Batch Multi-Agent Reinforcement Learning With Networked Agentscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 1; 2023: 3; 2024: 0; 2025: 0; 2026: 1.

7 papers.

  1. 2021A Dual Approach to Constrained Markov Decision Processes with Entropy Regularizationcited by 33
  2. 2022Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPscited by 26
  3. 2023A safe exploration approach to constrained Markov decision processescited by 10
  4. 2021Reward-Free Attacks in Multi-Agent Reinforcement Learningcited by 3
  5. 2023A New Policy Iteration Algorithm For Reinforcement Learning in Zero-Sum Markov Gamescited by 0
  6. 2023Reinforcement Learning With Reward Machines in Stochastic Gamescited by 0
  7. 2026Interactive Inverse Reinforcement Learning of Interaction Scenarios via Bi-level Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Data-Driven Inverse Reinforcement Learning for Expert-Learner Zero-Sum Games
  2. 2024On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning
  3. 2021Multi-Agent Inverse Reinforcement Learning: Suboptimal Demonstrations and Alternative Solution Concepts
  4. 2023Curricular Subgoals for Inverse Reinforcement Learning
  5. 2025Inverse Reinforcement Learning via Convex Optimization
  6. 2022Individual-Level Inverse Reinforcement Learning for Mean Field Games
  7. 2019Inverse Dynamic Games Based on Maximum Entropy Inverse Reinforcement Learning
  8. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress