# Convergence of Multi-Agent Learning with a Finite Step Size in General-Sum Games

arXiv: https://arxiv.org/abs/1903.02868 · HTML: https://mlsanity.com/paper/1903.02868/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2019.

## Built on by (most cited first, 8 of 8)

- [Policy Gradient Methods Find the Nash Equilibrium in N-player General-sum Linear-quadratic Games](https://mlsanity.com/paper/2107.13090/) (2021)
- [Coordinated Proximal Policy Optimization](https://mlsanity.com/paper/2111.04051/) (2021)
- [GemNet: Menu-Based, Strategy-Proof Multi-Bidder Auctions Through Deep Learning](https://mlsanity.com/paper/2406.07428/) (2024)
- [Multi-Sender Persuasion: A Computational Perspective](https://mlsanity.com/paper/2402.04971/) (2024)
- [The Bandit Whisperer: Communication Learning for Restless Bandits](https://mlsanity.com/paper/2408.05686/) (2024)
- [Deception in Social Learning: A Multi-Agent Reinforcement Learning Perspective](https://mlsanity.com/paper/2106.05402/) (2021)
- [Human Machine Co-Adaptation Model and Its Convergence Analysis](https://mlsanity.com/paper/2503.07319/) (2025)
- [LLM Active Alignment: A Nash Equilibrium Perspective](https://mlsanity.com/paper/2602.06836/) (2026)

## Abstract (arXiv)

Learning in a multi-agent system is challenging because agents are simultaneously learning and the environment is not stationary, undermining convergence guarantees. To address this challenge, this paper presents a new gradient-based learning algorithm, called Gradient Ascent with Shrinking Policy Prediction (GA-SPP), which augments the basic gradient ascent approach with the concept of shrinking policy prediction. The key idea behind this algorithm is that an agent adjusts its strategy in response to the forecasted strategy of the other agent, instead of its current one. GA-SPP is shown formally to have Nash convergence in larger settings than existing gradient-based multi-agent learning methods. Furthermore, unlike existing gradient-based methods, GA-SPP's theoretical guarantees do not assume the learning rate to be infinitesimal.