mlsanityData to 7 Oct 2026

Multi-Agent Training beyond Zero-Sum with Correlated Equilibrium Meta-Solvers

arXiv 2106.09435 ↗First posted 17 Jun 2021cs.MA cs.AI cs.GTcited by 21

Not on the map. Why?

Abstract (arXiv)

Two-player, constant-sum games are well studied in the literature, but there has been limited progress outside of this setting. We propose Joint Policy-Space Response Oracles (JPSRO), an algorithm for training agents in n-player, general-sum extensive form games, which provably converges to an equilibrium. We further suggest correlated equilibria (CE) as promising meta-solvers, and propose a novel solution concept Maximum Gini Correlated Equilibrium (MGCE), a principled and computationally efficient family of solutions for solving the correlated equilibrium selection problem. We conduct several experiments using CE meta-solvers for JPSRO and demonstrate convergence on n-player, general-sum games.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 22 of its 84 references that are in the map. First posted on arXiv 17 Jun 2021.

Builds on

22 of 84 references.

References · 22

  1. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  2. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  3. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
  4. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  5. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 180
  6. 2017A Rewriting System for Convex Optimization Problemscited by 178
  7. 2017OSQP: An Operator Splitting Solver for Quadratic Programscited by 156
  8. 2019Open-ended Learning in Symmetric Zero-sum Gamescited by 96
  9. 2019A Generalized Training Approach for Multiagent Learningcited by 53
  10. 2019α-Rank: Multi-Agent Evaluation by Evolutioncited by 50
  11. 2018Re-evaluating Evaluationcited by 47
  12. 2020Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Gamescited by 47
10 more
  1. 2020Human-Level Performance in No-Press Diplomacy via Equilibrium Searchcited by 41
  2. 2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
  3. 2020Learning to Play No-Press Diplomacy with Best Response Policy Iterationcited by 25
  4. 2019Correlation in Extensive-Form Games: Saddle-Point Formulation and Benchmarkscited by 23
  5. 2020Hindsight and Sequential Rationality of Correlated Playcited by 21
  6. 2020No-Regret Learning Dynamics for Extensive-Form Correlated Equilibriumcited by 19
  7. 2019Coarse Correlation in Extensive-Form Gamescited by 17
  8. 2021Solving Common-Payoff Games with Approximate Policy Iterationcited by 15
  9. 2019Learning to Correlate in Multi-Player General-Sum Sequential Gamescited by 9
  10. 2020Human-Agent Cooperation in Bridge Biddingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 8; 2023: 4; 2024: 5; 2025: 0; 2026: 2.

21 papers.

Sort:

By area:

  1. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  2. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  3. 2022NeuPL: Neural Population Learningcited by 14
  4. 2022Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computationcited by 14
  5. 2022Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Gamescited by 11
  6. 2022Turbocharging Solution Concepts: Solving NEs, CEs and CCEs with Neural Equilibrium Solverscited by 10
  7. 2024Policy Space Response Oracles: A Surveycited by 10
  8. 2021SoK: Machine Learning Governancecited by 9
  9. 2023Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learningcited by 6
  10. 2022Learning Correlated Equilibria in Mean-Field Gamescited by 4
  11. 2024Neural Population Learning beyond Symmetric Zero-sum Gamescited by 4
  12. 2022Game Theoretic Rating in N-player general-sum games with Equilibriacited by 3
9 more
  1. 2023Guarantees for Self-Play in Multiplayer Games via Polymatrix Decomposabilitycited by 3
  2. 2022Offline Equilibrium Findingcited by 2
  3. 2022Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environmentscited by 2
  4. 2023Are Equivariant Equilibrium Approximators Beneficial?cited by 2
  5. 2023Composing Efficient, Robust Tests for Policy Selectioncited by 1
  6. 2024Self-adaptive PSRO: Towards an Automatic Population-based Game Solvercited by 1
  7. 2024Configurable Mirror Descent: Towards a Unification of Decision Makingcited by 1
  8. 2026Conservative Equilibrium Discovery in Offline Game-Theoretic Multiagent Reinforcement Learningcited by 1
  9. 2026NePPO: Near-Potential Policy Optimization for General-Sum Multi-Agent Reinforcement Learningcited by 0