mlsanityData to 14 Oct 2026

Entropy-Regularized Mean-Field Games with Average-Reward Criterion: Span-Seminorm Contractions and Model-Free Learning

arXiv 2610.15871 ↗First posted 11 Oct 2026math.OC cs.GT cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity also fits: RL for games and combinatorial optimization

Abstract (arXiv)

We study stationary discrete-time mean-field games with finite state and action spaces under the average-reward criterion, with an entropy-regularized objective and population-dependent dynamics and rewards. Under a minorization condition, the soft Q-operator is a span-seminorm contraction for the full transition kernel, so relative value iteration does not require the minorizing measure. When the Lipschitz constant of the kernel is below the minorization mass and the temperature exceeds an explicit threshold, the equilibrium fixed-point operator is a contraction: an equilibrium exists, its population distribution is unique, and a fixed-point iteration converges to it geometrically. We then learn the equilibrium model-free from a simulator, combining soft Q-learning at a fixed population distribution with an empirical transition kernel. We prove a finite-sample bound for soft Q-learning in the span seminorm, with temperature-independent constants, and derive convergence rates for the learned population distribution and policy. A congestion-aware routing game illustrates the results.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 53 references that are in the map. First posted on arXiv 11 Oct 2026.

Builds on

23 of 53 references.

References · 23

  1. 2019A Theory of Regularized Markov Decision Processescited by 286
  2. 2017A unified view of entropy-regularized Markov decision processescited by 182
  3. 2017On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learningcited by 142
  4. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 73
  5. 2020Fictitious Play for Mean Field Games: Continuous Time Analysis and Applicationscited by 62
  6. 2022Learning in Mean Field Games: A Surveycited by 61
  7. 2020Entropy Regularization for Mean Field Games with Learningcited by 58
  8. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 49
  9. 2019On the Convergence of Model Free Learning in Mean Field Gamescited by 48
  10. 2016Markov-Nash Equilibria in Mean-Field Games with Discounted Costcited by 31
  11. 2022Oracle-free Reinforcement Learning in Mean-Field Games along a Single Sample Pathcited by 20
  12. 2019Actor-Critic Provably Finds Nash Equilibria of Linear-Quadratic Mean-Field Gamescited by 19
11 more
  1. 2020Exploratory LQG Mean Field Games with Entropy Regularizationcited by 15
  2. 2025A Non-Asymptotic Theory of Seminorm Lyapunov Stability: From Deterministic to Stochastic Iterative Algorithmscited by 10
  3. 2024When is Mean-Field Reinforcement Learning Tractable and Relevant?cited by 8
  4. 2014Linear-Quadratic N-person and Mean-Field Games with Ergodic Costcited by 5
  5. 2019Value Iteration Algorithm for Mean-field Gamescited by 5
  6. 2024Learning in Herding Mean Field Games: Single-Loop Algorithm with Finite-Time Convergence Analysiscited by 3
  7. 2015On Solutions of Mean Field Games with Ergodic Costcited by 2
  8. 2015Mean Field Games with Ergodic cost for Discrete Time Markov Processescited by 2
  9. 2019Discrete-time average-cost mean-field games on Polish spacescited by 2
  10. 2025Approximation of Discrete-Time Infinite-Horizon Mean-Field Equilibria via Finite-Horizon Mean-Field Equilibriacited by 1
  11. 2026Infinite Horizon Mean-Field Terminal Value Problemcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward
  2. 2025Kernel Based Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games
  3. 2024Unified continuous-time q-learning for mean-field game and mean-field control problems
  4. 2026Continuous-time q-learning for mean-field control with common noise, part-I: Theoretical foundations
  5. 2026On the Convergence of Semi-Gradient TD: A Mean Field Perspective
  6. 2026Achieving varepsilon-2 Dependence for Average-Reward Q-Learning with a New Contraction Principle
  7. 2026Continuous-Time Reinforcement Learning for N-Player Stochastic Differential Games with Exploratory Policies
  8. 2025Computing and Learning Stationary Mean Field Equilibria with Low-Dimensional Interactions: Algorithms and Applications ↗