JiangJun: Mastering Xiangqi by Tackling Non-Transitivity in Two-Player Zero-Sum Games
Not on the map. Why?
Abstract (arXiv)
This paper presents an empirical exploration of non-transitivity in perfect-information games, specifically focusing on Xiangqi, a traditional Chinese board game comparable in game-tree complexity to chess and shogi. By analyzing over 10,000 records of human Xiangqi play, we highlight the existence of both transitive and non-transitive elements within the game's strategic structure. To address non-transitivity, we introduce the JiangJun algorithm, an innovative combination of Monte-Carlo Tree Search (MCTS) and Policy Space Response Oracles (PSRO) designed to approximate a Nash equilibrium. We evaluate the algorithm empirically using a WeChat mini program and achieve a Master level with a 99.41% win rate against human players. The algorithm's effectiveness in overcoming non-transitivity is confirmed by a plethora of metrics, such as relative population performance and visualization results. Our project site is available at urlhttps://sites.google.com/view/jiangjun-site/.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 55 references that are in the map. First posted on arXiv 9 Aug 2023.
Builds on
30 of 55 references.
References · 30
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
- 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
- 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 308
- 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
- 2021Monte Carlo Tree Search: A Review of Recent Modifications and Applicationscited by 158
- 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 153
- 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 129
- 2018Deep Counterfactual Regret Minimizationcited by 102
- 2019Open-ended Learning in Symmetric Zero-sum Gamescited by 96
- 2020Real World Games Look Like Spinning Topscited by 66
- 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
18 more
- 2020From Poincar'e Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularizationcited by 59
- 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 55
- 2019A Generalized Training Approach for Multiagent Learningcited by 53
- 2020Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Gamescited by 47
- 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
- 2020DREAM: Deep Regret minimization with Advantage baselines and Model-free learningcited by 36
- 2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
- 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
- 2022ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regretcited by 24
- 2020Approximate exploitability: Learning a best response in large gamescited by 23
- 2022PerfectDou: Dominating DouDizhu with Perfect Information Distillationcited by 15
- 2021Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Gamescited by 14
- 2019Neural Replicator Dynamicscited by 13
- 2018Hierarchical Macro Strategy Model for MOBA Game AIcited by 12
- 2022Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Gamescited by 11
- 2019Action Guidance with MCTS for Deep Reinforcement Learningcited by 8
- 2019Learning Policies from Self-Play with Policy Gradients and MCTS Value Estimatescited by 6
- 2022Feasible Adversarial Robust Reinforcement Learning for Underspecified Environmentscited by 4
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.