# Learning Distributed Equilibria in Linear-Quadratic Stochastic Differential Games: An α-Potential Approach

arXiv: https://arxiv.org/abs/2602.16555 · HTML: https://mlsanity.com/paper/2602.16555/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 18 Feb 2026.

## Builds on (most cited first, 22 of 22)

- [A Survey and Critique of Multiagent Deep Reinforcement Learning](https://mlsanity.com/paper/1810.05587/) (2018)
- [Global Convergence of Multi-Agent Policy Gradient in Markov Potential Games](https://mlsanity.com/paper/2106.01969/) (2021)
- [A Theoretical and Empirical Comparison of Gradient Approximations in Derivative-Free Optimization](https://mlsanity.com/paper/1905.01332/) (2019)
- [Learning in Mean Field Games: A Survey](https://mlsanity.com/paper/2205.12944/) (2022)
- [Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergence](https://mlsanity.com/paper/2202.04129/) (2022)
- [Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysis](https://mlsanity.com/paper/2002.04131/) (2020)
- [Independent Natural Policy Gradient Always Converges in Markov Potential Games](https://mlsanity.com/paper/2110.10614/) (2021)
- [Policy Gradient Methods Find the Nash Equilibrium in N-player General-sum Linear-quadratic Games](https://mlsanity.com/paper/2107.13090/) (2021)
- [Convergence of policy gradient methods for finite-horizon exploratory linear-quadratic control problems](https://mlsanity.com/paper/2211.00617/) (2022)
- [Independent and Decentralized Learning in Markov Potential Games](https://mlsanity.com/paper/2205.14590/) (2022)
- [Policy-Gradient Algorithms Have No Guarantees of Convergence in Linear Quadratic Games](https://mlsanity.com/paper/1907.03712/) (2019)
- [Independent Learning in Constrained Markov Potential Games](https://mlsanity.com/paper/2402.17885/) (2024)
- [A case study on stochastic games on large graphs in mean field and sparse regimes](https://mlsanity.com/paper/2005.14102/) (2020)
- [Synchronization in a Kuramoto Mean Field Game](https://mlsanity.com/paper/2210.12912/) (2022)
- [Markov Games with Decoupled Dynamics: Price of Anarchy and Sample Complexity](https://mlsanity.com/paper/2304.03840/) (2023)
- [A Cucker-Smale inspired deterministic mean field game with velocity interactions](https://mlsanity.com/paper/2009.07519/) (2020)
- [Approximately optimal distributed stochastic controls beyond the mean field setting](https://mlsanity.com/paper/2301.02901/) (2023)
- [On Linear Quadratic Potential Games](https://mlsanity.com/paper/2305.13476/) (2023)
- [Towards An Analytical Framework for Dynamic Potential Games](https://mlsanity.com/paper/2310.02259/) (2023)
- [Policy Optimization for Continuous-time Linear-Quadratic Graphon Mean Field Games](https://mlsanity.com/paper/2506.05894/) (2025)
- [Asymmetric Network Games: α-Potential Function and Learning](https://mlsanity.com/paper/2508.06619/) (2025)
- [α-Potential Games for Decentralized Control of Connected and Automated Vehicles](https://mlsanity.com/paper/2512.05712/) (2025)

## Built on by (most cited first, 2 of 2)

- [Continuous-Time Reinforcement Learning for N-Player Stochastic Differential Games with Exploratory Policies](https://mlsanity.com/paper/2607.19928/) (2026)
- [Learning under Opponent Unawareness in Linear-Quadratic Stochastic Games](https://mlsanity.com/paper/2608.08268/) (2026)

## Abstract (arXiv)

We analyze independent policy-gradient (PG) learning in N-player linear-quadratic (LQ) stochastic differential games. Each player employs a distributed policy that depends only on its own state and updates the policy independently using the gradient of its own objective. We establish global linear convergence of these methods to an equilibrium by showing that the LQ game admits an α-potential structure, with α determined by the degree of pairwise interaction asymmetry. For pairwise-symmetric interactions, we construct an affine distributed equilibrium by minimizing the potential function and show that independent PG methods converge globally to this equilibrium, with complexity scaling linearly in the population size and logarithmically in the desired accuracy. For asymmetric interactions, we prove that independent projected PG algorithms converge linearly to an approximate equilibrium, with suboptimality proportional to the degree of asymmetry. Numerical experiments confirm the theoretical results across both symmetric and asymmetric interaction networks.