mlsanityData to 14 Oct 2026

Maximum Causal Entropy IRL in Mean-Field Games and GNEP Framework for Forward RL

arXiv 2401.06566 ↗First posted 12 Jan 2024eess.SY cs.LG cs.SYcited by 4

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

This paper explores the use of Maximum Causal Entropy Inverse Reinforcement Learning (IRL) within the context of discrete-time stationary Mean-Field Games (MFGs) characterized by finite state spaces and an infinite-horizon, discounted-reward setting. Although the resulting optimization problem is non-convex with respect to policies, we reformulate it as a convex optimization problem in terms of state-action occupation measures by leveraging the linear programming framework of Markov Decision Processes. Based on this convex reformulation, we introduce a gradient descent algorithm with a guaranteed convergence rate to efficiently compute the optimal solution. Moreover, we develop a new method that conceptualizes the MFG problem as a Generalized Nash Equilibrium Problem (GNEP), enabling effective computation of the mean-field equilibrium for forward reinforcement learning (RL) problems and marking an advancement in MFG solution techniques. We further illustrate the practical applicability of our GNEP approach by employing this algorithm to generate data for numerical MFG examples.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 13 of its 36 references that are in the map. First posted on arXiv 12 Jan 2024.

Builds on

13 of 36 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023On Imitation in Mean-field Games26 Jun
  2. 2022A Primer on Maximum Causal Entropy Inverse Reinforcement Learning22 Mar
  3. 2022Individual-Level Inverse Reinforcement Learning for Mean Field Games13 Feb
  4. 2021Adversarial Inverse Reinforcement Learning for Mean Field Games29 Apr
  5. 2020Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms1 Dec

Other references · 8

  1. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  2. 2017A unified view of entropy-regularized Markov decision processescited by 182
  3. 2023Handbook of Convergence Theorems for (Stochastic) Gradient Methodscited by 163
  4. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 73
  5. 2022Learning in Mean Field Games: A Surveycited by 61
  6. 2020Q-Learning in Regularized Mean-field Gamescited by 42
  7. 2017Learning Deep Mean Field Games for Modeling Large Population Behaviorcited by 8
  8. 2023Linear Mean-Field Games with Discounted Costcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Kernel Based Maximum Entropy Inverse Reinforcement Learning for Mean-Field Gamescited by 3
  2. 2026Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Rewardcited by 0
  3. 2026Neural Parameter Calibration for Finite-State Mean Field Gamescited by 0
  4. 2026Inverse Learning of the Altruism and Cost Level in Mixed-Individual Mean Field Gamescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Generalized Maximum Causal Entropy for Inverse Reinforcement Learning
  2. 2019Inverse Dynamic Games Based on Maximum Entropy Inverse Reinforcement Learning
  3. 2026Fast Rates for Inverse Reinforcement Learning
  4. 2026Inverse reinforcement learning for indefinite mean-field social optimization with multiplicative noise
  5. 2021A Generalised Inverse Reinforcement Learning Framework
  6. 2021Inverse Reinforcement Learning with Explicit Policy Estimates
  7. 2024Independent RL for Cooperative-Competitive Agents: A Mean-Field Perspective
  8. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch