# Efficient computation of rankings from pairwise comparisons

arXiv: https://arxiv.org/abs/2207.00076 · HTML: https://mlsanity.com/paper/2207.00076/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 27 references that are in the map.
- First posted on arXiv 30 Jun 2022.

## Builds on (most cited first, 1 of 1)

- [IMDB-WIKI-SbS: An Evaluation Dataset for Crowdsourced Pairwise Comparisons](https://mlsanity.com/paper/2110.14990/) (2021)

## Built on by (most cited first, 30 of 31)

- [Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support Conversation](https://mlsanity.com/paper/2402.13211/) (2024)
- [Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisons](https://mlsanity.com/paper/2405.05894/) (2024)
- [A Statistical Framework for Ranking LLM-Based Chatbots](https://mlsanity.com/paper/2412.18407/) (2024)
- [Finding the Subjective Truth: Collecting 2 Million Votes for Comprehensive Gen-AI Model Evaluation](https://mlsanity.com/paper/2409.11904/) (2024)
- [AutoPersuade: A Framework for Evaluating and Explaining Persuasive Arguments](https://mlsanity.com/paper/2410.08917/) (2024)
- [On Sinkhorn's Algorithm and Choice Modeling](https://mlsanity.com/paper/2310.00260/) (2023)
- [Luck, skill, and depth of competition in games and social hierarchies](https://mlsanity.com/paper/2312.04711/) (2023)
- [Statistical ranking with dynamic covariates](https://mlsanity.com/paper/2406.16507/) (2024)
- [Generalised Probabilistic Modelling and Improved Uncertainty Estimation in Comparative LLM-as-a-judge](https://mlsanity.com/paper/2505.15240/) (2025)
- [Confidence and Stability of Global and Pairwise Scores in NLP Evaluation](https://mlsanity.com/paper/2507.01633/) (2025)
- [Statistical inference for pairwise comparison models](https://mlsanity.com/paper/2401.08463/) (2024)
- [Reliable, Reproducible, and Really Fast Leaderboards with Evalica](https://mlsanity.com/paper/2412.11314/) (2024)
- [Estimation of partial rankings from sparse, noisy comparisons](https://mlsanity.com/paper/2501.02505/) (2025)
- [C2-DPO: Constrained Controlled Direct Preference Optimization](https://mlsanity.com/paper/2502.17507/) (2025)
- [Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling](https://mlsanity.com/paper/2508.16876/) (2025)
- [Interestingness First Classifiers](https://mlsanity.com/paper/2508.19780/) (2025)
- [Convergence analysis of a family of Zermelo-type iterations for the Bradley--Terry model](https://mlsanity.com/paper/2607.22221/) (2026)
- [iREPO: implicit Reward Pairwise Difference based Empirical Preference Optimization](https://mlsanity.com/paper/2405.15230/) (2024)
- [Absolute Ranking: An Essential Normalization for Benchmarking Optimization Algorithms](https://mlsanity.com/paper/2409.04479/) (2024)
- [Ranking of Multi-Response Experiment Treatments](https://mlsanity.com/paper/2410.17604/) (2024)
- [Efficient Portfolio Selection through Preference Aggregation with Quicksort and the Bradley--Terry Model](https://mlsanity.com/paper/2504.16093/) (2025)
- [Estimating the Error of Large Language Models at Pairwise Text Comparison](https://mlsanity.com/paper/2510.22219/) (2025)
- [Distribution-informed Efficient Conformal Prediction for Full Ranking](https://mlsanity.com/paper/2601.23128/) (2026)
- [On rankings in multiplayer games with an application to the game of Whist](https://mlsanity.com/paper/2604.00641/) (2026)
- [New insights into Elo algorithm for practitioners and statisticians](https://mlsanity.com/paper/2604.03840/) (2026)
- [Calibrated Preference Learning: The Case of Label Ranking](https://mlsanity.com/paper/2605.30447/) (2026)
- [Pairwise Reference Alignment as a Model-Level Ordinal Observable](https://mlsanity.com/paper/2605.30758/) (2026)
- [ToSCA: Leveraging Hierarchical Reinforcement Learning on Temporal and Strategic Abstractions of Conversational Agents](https://mlsanity.com/paper/2608.21969/) (2026)
- [CoArena: Evaluating Computer-Use and Multi-Agent Systems in Real Time](https://mlsanity.com/paper/2609.14239/) (2026)
- [Steering the Compass: Aligning Dynamic Psychological Counseling Conversations with Cognitive Behavioral Therapy Strategies](https://mlsanity.com/paper/2609.20565/) (2026)

## Abstract (arXiv)

We study the ranking of individuals, teams, or objects, based on pairwise comparisons between them, using the Bradley-Terry model. Estimates of rankings within this model are commonly made using a simple iterative algorithm first introduced by Zermelo almost a century ago. Here we describe an alternative and similarly simple iteration that provably returns identical results but does so much faster -- over a hundred times faster in some cases. We demonstrate this algorithm with applications to a range of example data sets and derive a number of results regarding its convergence.