# TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2011.12895 · HTML: https://mlsanity.com/paper/2011.12895/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2020.

## Built on by (most cited first, 11 of 11)

- [Neural Categorical Priors for Physics-Based Character Control](https://mlsanity.com/paper/2308.07200/) (2023)
- [Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models](https://mlsanity.com/paper/2308.15143/) (2023)
- [MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2106.07551/) (2021)
- [Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Play](https://mlsanity.com/paper/2303.04096/) (2023)
- [Mastering Strategy Card Game (Hearthstone) with Improved Techniques](https://mlsanity.com/paper/2303.05197/) (2023)
- [SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores](https://mlsanity.com/paper/2306.16688/) (2023)
- [Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning towards Asynchronous Commercial Games](https://mlsanity.com/paper/2209.10055/) (2022)
- [MSRL: Distributed Reinforcement Learning with Dataflow Fragments](https://mlsanity.com/paper/2210.00882/) (2022)
- [SVDE: Scalable Value-Decomposition Exploration for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.09058/) (2023)
- [CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community](https://mlsanity.com/paper/2407.06485/) (2024)
- [Cooperative-Competitive Team Play of Real-World Craft Robots](https://mlsanity.com/paper/2602.21119/) (2026)

## Abstract (arXiv)

Competitive Self-Play (CSP) based Multi-Agent Reinforcement Learning (MARL) has shown phenomenal breakthroughs recently. Strong AIs are achieved for several benchmarks, including Dota 2, Glory of Kings, Quake III, StarCraft II, to name a few. Despite the success, the MARL training is extremely data thirsty, requiring typically billions of (if not trillions of) frames be seen from the environment during training in order for learning a high performance agent. This poses non-trivial difficulties for researchers or engineers and prevents the application of MARL to a broader range of real-world problems. To address this issue, in this manuscript we describe a framework, referred to as TLeague, that aims at large-scale training and implements several main-stream CSP-MARL algorithms. The training can be deployed in either a single machine or a cluster of hybrid machines (CPUs and GPUs), where the standard Kubernetes is supported in a cloud native manner. TLeague achieves a high throughput and a reasonable scale-up when performing distributed training. Thanks to the modular design, it is also easy to extend for solving other multi-agent problems or implementing and verifying MARL algorithms. We present experiments over StarCraft II, ViZDoom and Pommerman to show the efficiency and effectiveness of TLeague. The code is open-sourced and available at https://github.com/tencent-ailab/tleague_projpage