mlsanityData to 14 Oct 2026

Vairiational Stochastic Games

arXiv 2503.06037 ↗First posted 8 Mar 2025cs.MA cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

The Control as Inference (CAI) framework has successfully transformed single-agent reinforcement learning (RL) by reframing control tasks as probabilistic inference problems. However, the extension of CAI to multi-agent, general-sum stochastic games (SGs) remains underexplored, particularly in decentralized settings where agents operate independently without centralized coordination. In this paper, we propose a novel variational inference framework tailored to decentralized multi-agent systems. Our framework addresses the challenges posed by non-stationarity and unaligned agent objectives, proving that the resulting policies form an ε-Nash equilibrium. Additionally, we demonstrate theoretical convergence guarantees for the proposed decentralized algorithms. Leveraging this framework, we instantiate multiple algorithms to solve for Nash equilibrium, mean-field Nash equilibrium, and correlated equilibrium, with rigorous theoretical convergence analysis.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 29 references that are in the map. First posted on arXiv 8 Mar 2025.

Builds on

14 of 29 references.

References · 14

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2016Variational Inference: A Review for Statisticianscited by 1,472
  3. 2014Convex Optimization: Algorithms and Complexitycited by 575
  4. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  5. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  6. 2015Taming the Noise in Reinforcement Learning via Soft Updatescited by 127
  7. 2016Opponent Modeling in Deep Reinforcement Learningcited by 97
  8. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 73
  9. 2019Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learningcited by 68
  10. 2018Modeling Others using Oneself in Multi-Agent Reinforcement Learningcited by 66
  11. 2018Multiagent Soft Q-Learningcited by 26
  12. 2021Learning in Nonzero-Sum Stochastic Games with Potentialscited by 26
2 more
  1. 2019Modelling Bounded Rationality in Multi-Agent Interactions by Generalized Recursive Reasoningcited by 24
  2. 2019A Regularized Opponent Model with Maximum Entropy Objectivecited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Convergence of Decentralized Actor-Critic Algorithm in General-sum Markov Games
  2. 2024Independent RL for Cooperative-Competitive Agents: A Mean-Field Perspective
  3. 2026Learning Approximate Nash Equilibria in Cooperative Multi-Agent Reinforcement Learning via Mean-Field Subsampling
  4. 2026Actor-Dual-Critic Dynamics for Zero-sum and Identical-Interest Stochastic Games
  5. 2024Equilibrium Selection for Multi-agent Reinforcement Learning: A Unified Framework
  6. 2018Multi-agent Inverse Reinforcement Learning for Certain General-sum Stochastic Games
  7. 2024Game Theory and Multi-Agent Reinforcement Learning : From Nash Equilibria to Evolutionary Dynamics
  8. 2024MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Games