# Elo Ratings for Large Tournaments of Software Agents in Asymmetric Games

arXiv: https://arxiv.org/abs/2105.00839 · HTML: https://mlsanity.com/paper/2105.00839/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 13 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 1 of 1)

- [Modelling Competitive Sports: Bradley-Terry-'ElHo Models for Supervised and On-Line Learning of Paired Competition Outcomes](https://mlsanity.com/paper/1701.08055/) (2017)

## Built on by (most cited first, 2 of 2)

- [Elo Uncovered: Robustness and Best Practices in Language Model Evaluation](https://mlsanity.com/paper/2311.17295/) (2023)
- [AREG: Adversarial Resource Extraction Game for Evaluating Persuasion and Resistance in Large Language Models](https://mlsanity.com/paper/2602.16639/) (2026)

## Abstract (arXiv)

The Elo rating system has been used world wide for individual sports and team sports, as exemplified by the European Go Federation (EGF), International Chess Federation (FIDE), International Federation of Association Football (FIFA), and many others. To evaluate the performance of artificial intelligence agents, it is natural to evaluate them on the same Elo scale as humans, such as the rating of 5185 attributed to AlphaGo Zero. There are several fundamental differences between humans and AI that suggest modifications to the system, which in turn require revisiting Elo's fundamental rationale. AI is typically trained on many more games than humans play, and we have little a-priori information on newly created AI agents. Further, AI is being extended into games which are asymmetric between the players, and which could even have large complex boards with different setup in every game, such as commercial paper strategy games. We present a revised rating system, and guidelines for tournaments, to reflect these differences.