mlsanityData to 7 Oct 2026

Data-Driven Inverse Reinforcement Learning for Expert-Learner Zero-Sum Games

arXiv 2301.01997 ↗First posted 5 Jan 2023cs.LG cs.SY eess.SYcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

In this paper, we formulate inverse reinforcement learning (IRL) as an expert-learner interaction whereby the optimal performance intent of an expert or target agent is unknown to a learner agent. The learner observes the states and controls of the expert and hence seeks to reconstruct the expert's cost function intent and thus mimics the expert's optimal response. Next, we add non-cooperative disturbances that seek to disrupt the learning and stability of the learner agent. This leads to the formulation of a new interaction we call zero-sum game IRL. We develop a framework to solve the zero-sum game IRL problem that is a modified extension of RL policy iteration (PI) to allow unknown expert performance intentions to be computed and non-cooperative disturbances to be rejected. The framework has two parts: a value function and control action update based on an extension of PI, and a cost function update based on standard inverse optimal control. Then, we eventually develop an off-policy IRL algorithm that does not require knowledge of the expert and learner agent dynamics and performs single-loop learning. Rigorous proofs and analyses are given. Finally, simulation experiments are presented to show the effectiveness of the new approach.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 4 of its 42 references that are in the map. First posted on arXiv 5 Jan 2023.

Builds on

4 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2020Online Observer-Based Inverse Reinforcement Learning3 Nov
  2. 2020Online inverse reinforcement learning with unknown disturbances9 Mar

Other references · 2

  1. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  2. 2020Online Inverse Optimal Control for Control-Constrained Discrete-Time Systems on Finite and Infinite Horizonscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  2. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  3. 2024Inverse Reinforcement Learning with Sub-optimal Experts
  4. 2018Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations
  5. 2026Interactive Inverse Reinforcement Learning of Interaction Scenarios via Bi-level Optimization
  6. 2024Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms
  7. 2024Hybrid Inverse Reinforcement Learning
  8. 2025Inverse Reinforcement Learning via Convex Optimization