mlsanityData to 7 Oct 2026

Decentralized Q-Learning for Stochastic Teams and Games

arXiv 1506.07924 ↗First posted 25 Jun 2015math.OC cs.GT cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

There are only a few learning algorithms applicable to stochastic dynamic teams and games which generalize Markov decision processes to decentralized stochastic control problems involving possibly self-interested decision makers. Learning in games is generally difficult because of the non-stationary environment in which each decision maker aims to learn its optimal decisions with minimal information in the presence of the other decision makers who are also learning. In stochastic dynamic games, learning is more challenging because, while learning, the decision makers alter the state of the system and hence the future cost. In this paper, we present decentralized Q-learning algorithms for stochastic games, and study their convergence for the weakly acyclic case which includes team problems as an important special case. The algorithm is decentralized in that each decision maker has access to only its local information, the state information, and the local cost realizations; furthermore, it is completely oblivious to the presence of other decision makers. We show that these algorithms converge to equilibrium policies almost surely in large classes of stochastic games.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 6; 2023: 6; 2024: 4; 2025: 6; 2026: 1.

34 papers.

Sort:

  1. 2021Independent Policy Gradient Methods for Competitive Reinforcement Learningcited by 97
  2. 2021Decentralized Q-Learning in Zero-sum Markov Gamescited by 53
  3. 2021Provably Efficient Reinforcement Learning in Decentralized General-Sum Markov Gamescited by 42
  4. 2021On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learningcited by 35
  5. 2021Gradient play in stochastic games: stationary points, convergence, and sample complexitycited by 32
  6. 2023Uncoupled and Convergent Learning in Two-Player Zero-Sum Markov Games with Bandit Feedbackcited by 28
  7. 2021Multi-Agent Reinforcement Learning with Temporal Logic Specificationscited by 27
  8. 2021Independent Learning in Stochastic Gamescited by 20
  9. 2022Independent and Decentralized Learning in Markov Potential Gamescited by 19
  10. 2022Oracle-free Reinforcement Learning in Mean-Field Games along a Single Sample Pathcited by 16
  11. 2023A Finite-Sample Analysis of Payoff-Based Independent Learning in Zero-Sum Stochastic Gamescited by 15
  12. 2022Independent Natural Policy Gradient Methods for Potential Games: Finite-time Global Convergence with Entropy Regularizationcited by 10
22 more
  1. 2021Satisficing Paths and Independent Multi-Agent Reinforcement Learning in Stochastic Gamescited by 9
  2. 2021MDPGT: Momentum-based Decentralized Policy Gradient Trackingcited by 6
  3. 2023Markov α-Potential Gamescited by 6
  4. 2021Provably Efficient Multi-Agent Reinforcement Learning with Fully Decentralized Communicationcited by 5
  5. 2023Q-Learning for Stochastic Control under General Information Structures and Non-Markovian Environmentscited by 5
  6. 2024Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Surveycited by 5
  7. 2021Finite-Sample Analysis of Decentralized Q-Learning for Stochastic Gamescited by 4
  8. 2023Unsynchronized Decentralized Q-Learning: Two Timescale Analysis By Persistencecited by 3
  9. 2024Convergence of Decentralized Actor-Critic Algorithm in General-sum Markov Gamescited by 2
  10. 2024Paths to Equilibrium in Gamescited by 1
  11. 2022Deep Q-learning of global optimizer of multiply model parameters for viscoelastic imagingcited by 0
  12. 2022Deep Reinforcement Learning for Distributed and Uncoordinated Cognitive Radios Resource Allocationcited by 0
  13. 2022Knowing the Past to Predict the Future: Reinforcement Virtual Learningcited by 0
  14. 2023Decentralized Multi-Agent Reinforcement Learning for Continuous-Space Stochastic Gamescited by 0
  15. 2024The Bounds of Algorithmic Collusion; Q-learning, Gradient Learning, and the Folk Theoremcited by 0
  16. 2025Q-MARL: A quantum-inspired algorithm using neural message passing for large-scale multi-agent reinforcement learningcited by 0
  17. 2025Policy Optimization and Multi-agent Reinforcement Learning for Mean-variance Team Stochastic Gamescited by 0
  18. 2025Finite-Time Global Optimality Convergence in Deep Neural Actor-Critic Methods for Decentralized Multi-Agent Reinforcement Learningcited by 0
  19. 2025Grouped Satisficing Paths in Pure Strategy Games: a Topological Perspectivecited by 0
  20. 2025Local Reinforcement Learning with Action-Conditioned Root Mean Squared Q-Functionscited by 0
  21. 2025Aspiration-based Perturbed Learning Automata in Games with Noisy Utility Measurements. Part A: Stochastic Stability in Non-zero-Sum Gamescited by 0
  22. 2026Actor-Dual-Critic Dynamics for Zero-sum and Identical-Interest Stochastic Gamescited by 0