mlsanityData to 14 Oct 2026

Neural Population Learning beyond Symmetric Zero-sum Games

arXiv 2401.05133 ↗First posted 10 Jan 2024cs.AI cs.MAcited by 4

Not on the map. Why?

Abstract (arXiv)

We study computationally efficient methods for finding equilibria in n-player general-sum games, specifically ones that afford complex visuomotor skills. We show how existing methods would struggle in this setting, either computationally or in theory. We then introduce NeuPL-JPSRO, a neural population learning algorithm that benefits from transfer learning of skills and converges to a Coarse Correlated Equilibrium (CCE) of the game. We show empirical convergence in a suite of OpenSpiel games, validated rigorously by exact game solvers. We then deploy NeuPL-JPSRO to complex domains, where our approach enables adaptive coordination in a MuJoCo control domain and skill transfer in capture-the-flag. Our work shows that equilibrium convergent population learning can be implemented at scale and in generality, paving the way towards solving real-world games between heterogeneous players with mixed motives.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 24 of its 45 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

24 of 45 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  3. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  4. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  5. 2018Maximum a Posteriori Policy Optimisationcited by 328
  6. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  7. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 264
  8. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
  9. 2019OpenSpiel: A Framework for Reinforcement Learning in Gamescited by 186
  10. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 99
  11. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  12. 2021Near-Optimal No-Regret Learning in General Gamescited by 85
12 more
  1. 2020From Poincar'e Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularizationcited by 61
  2. 2019Emergent Coordination Through Competitioncited by 57
  3. 2019A Generalized Training Approach for Multiagent Learningcited by 54
  4. 2019α-Rank: Multi-Agent Evaluation by Evolutioncited by 51
  5. 2020Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Gamescited by 47
  6. 2019Correlation in Extensive-Form Games: Saddle-Point Formulation and Benchmarkscited by 23
  7. 2021Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversitycited by 22
  8. 2021Multi-Agent Training beyond Zero-Sum with Correlated Equilibrium Meta-Solverscited by 21
  9. 2021Iterative Empirical Game Solving via Single Policy Best Responsecited by 12
  10. 2022Turbocharging Solution Concepts: Solving NEs, CEs and CCEs with Neural Equilibrium Solverscited by 10
  11. 2022Revisiting Gaussian mixture critics in off-policy reinforcement learning: a sample-based approachcited by 6
  12. 2015Finding Any Nontrivial Coarse Correlated Equilibrium Is Hardcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 2.

4 papers.

  1. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  2. 2024Policy Space Response Oracles: A Surveycited by 10
  3. 2026Sample-Efficient Policy Space Response Oracles with Joint Experience Best Responsecited by 0
  4. 2026Towards Learning Representations of Policies in Two-Player Zero-Sum Imperfect-Information Gamescited by 0