# Generalized Individual Q-learning for Polymatrix Games with Partial Observations

arXiv: https://arxiv.org/abs/2409.02663 · HTML: https://mlsanity.com/paper/2409.02663/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 3 of its 15 references that are in the map.
- First posted on arXiv 4 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 3 of 3)

- [On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learning](https://mlsanity.com/paper/1704.00805/) (2017)
- [Multi-Player Zero-Sum Markov Games with Networked Separable Interactions](https://mlsanity.com/paper/2307.09470/) (2023)
- [Convergence of Heterogeneous Learning Dynamics in Zero-sum Stochastic Games](https://mlsanity.com/paper/2311.00778/) (2023)

## Built on by (most cited first, 1 of 1)

- [Aggregate Fictitious Play for Learning in Anonymous Polymatrix Games (Extended Version)](https://mlsanity.com/paper/2508.19371/) (2025)

## Abstract (arXiv)

This paper addresses the challenge of limited observations in non-cooperative multi-agent systems where agents can have partial access to other agents' actions. We present the generalized individual Q-learning dynamics that combine belief-based and payoff-based learning for the networked interconnections of more than two self-interested agents. This approach leverages access to opponents' actions whenever possible, demonstrably achieving a faster (guaranteed) convergence to quantal response equilibrium in multi-agent zero-sum and potential polymatrix games. Notably, the dynamics reduce to the well-studied smoothed fictitious play and individual Q-learning under full and no access to opponent actions, respectively. We further quantify the improvement in convergence rate due to observing opponents' actions through numerical simulations.