# Suphx: Mastering Mahjong with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2003.13590 · HTML: https://mlsanity.com/paper/2003.13590/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2020.

## Built on by (most cited first, 30 of 58)

- [DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning](https://mlsanity.com/paper/2106.06135/) (2021)
- [Pgx: Hardware-Accelerated Parallel Game Simulators for Reinforcement Learning](https://mlsanity.com/paper/2303.17503/) (2023)
- [What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?](https://mlsanity.com/paper/2305.18882/) (2023)
- [Universal Trading for Order Execution with Oracle Policy Distillation](https://mlsanity.com/paper/2103.10860/) (2021)
- [A Survey on Self-play Methods in Reinforcement Learning](https://mlsanity.com/paper/2408.01072/) (2024)
- [ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regret](https://mlsanity.com/paper/2206.04122/) (2022)
- [RetroGraph: Retrosynthetic Planning with Graph Search](https://mlsanity.com/paper/2206.11477/) (2022)
- [Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemble](https://mlsanity.com/paper/2205.09284/) (2022)
- [PerfectDou: Dominating DouDizhu with Perfect Information Distillation](https://mlsanity.com/paper/2203.16406/) (2022)
- [FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2406.02081/) (2024)
- [PUZZLES: A Benchmark for Neural Algorithmic Reasoning](https://mlsanity.com/paper/2407.00401/) (2024)
- [Trust Region Preference Approximation: A simple and stable reinforcement learning algorithm for LLM reasoning](https://mlsanity.com/paper/2504.04524/) (2025)
- [Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RL](https://mlsanity.com/paper/2207.05683/) (2022)
- [Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Play](https://mlsanity.com/paper/2303.04096/) (2023)
- [Mastering Strategy Card Game (Hearthstone) with Improved Techniques](https://mlsanity.com/paper/2303.05197/) (2023)
- [Bridging the Sim-to-Real Gap from the Information Bottleneck Perspective](https://mlsanity.com/paper/2305.18464/) (2023)
- [DouZero+: Improving DouDizhu AI by Opponent Modeling and Coach-guided Learning](https://mlsanity.com/paper/2204.02558/) (2022)
- [PIGDreamer: Privileged Information Guided World Models for Safe Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2508.02159/) (2025)
- [DanZero+: Dominating the GuanDan Game through Reinforcement Learning](https://mlsanity.com/paper/2312.02561/) (2023)
- [UNO Arena for Evaluating Sequential Decision-Making Capability of Large Language Models](https://mlsanity.com/paper/2406.16382/) (2024)
- [DanZero: Mastering GuanDan Game with Reinforcement Learning](https://mlsanity.com/paper/2210.17087/) (2022)
- [Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolbox](https://mlsanity.com/paper/2212.00253/) (2022)
- [Can Large Language Models Master Complex Card Games?](https://mlsanity.com/paper/2509.01328/) (2025)
- [A survey on algorithms for Nash equilibria in finite normal-form games](https://mlsanity.com/paper/2312.11063/) (2023)
- [Goal-Directed Planning by Reinforcement Learning and Active Inference](https://mlsanity.com/paper/2106.09938/) (2021)
- [Building a 3-Player Mahjong AI using Deep Reinforcement Learning](https://mlsanity.com/paper/2202.12847/) (2022)
- [JiangJun: Mastering Xiangqi by Tackling Non-Transitivity in Two-Player Zero-Sum Games](https://mlsanity.com/paper/2308.04719/) (2023)
- [RL-LLM-DT: An Automatic Decision Tree Generation Method Based on RL Evaluation and LLM Enhancement](https://mlsanity.com/paper/2412.11417/) (2024)
- [AI in Human-computer Gaming: Techniques, Challenges and Opportunities](https://mlsanity.com/paper/2111.07631/) (2021)
- [Efficient Distributed Framework for Collaborative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.05248/) (2022)

## Abstract (arXiv)

Artificial Intelligence (AI) has achieved great success in many domains, and game AI is widely regarded as its beachhead since the dawn of AI. In recent years, studies on game AI have gradually evolved from relatively simple environments (e.g., perfect-information games such as Go, chess, shogi or two-player imperfect-information games such as heads-up Texas hold'em) to more complex ones (e.g., multi-player imperfect-information games such as multi-player Texas hold'em and StartCraft II). Mahjong is a popular multi-player imperfect-information game worldwide but very challenging for AI research due to its complex playing/scoring rules and rich hidden information. We design an AI for Mahjong, named Suphx, based on deep reinforcement learning with some newly introduced techniques including global reward prediction, oracle guiding, and run-time policy adaptation. Suphx has demonstrated stronger performance than most top human players in terms of stable rank and is rated above 99.99% of all the officially ranked human players in the Tenhou platform. This is the first time that a computer program outperforms most top human players in Mahjong.