mlsanityData to 7 Oct 2026

Efficient computation of rankings from pairwise comparisons

arXiv 2207.00076 ↗First posted 30 Jun 2022stat.ML cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

We study the ranking of individuals, teams, or objects, based on pairwise comparisons between them, using the Bradley-Terry model. Estimates of rankings within this model are commonly made using a simple iterative algorithm first introduced by Zermelo almost a century ago. Here we describe an alternative and similarly simple iteration that provably returns identical results but does so much faster -- over a hundred times faster in some cases. We demonstrate this algorithm with applications to a range of example data sets and derive a number of results regarding its convergence.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 1 of its 27 references that are in the map. First posted on arXiv 30 Jun 2022.

Builds on

1 of 27 references.

References · 1

  1. 2021IMDB-WIKI-SbS: An Evaluation Dataset for Crowdsourced Pairwise Comparisonscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 11; 2025: 8; 2026: 10.

31 papers.

Sort:

By area:

  1. 2024Can Large Language Models be Good Emotional Supporter? Mitigating Preference Bias on Emotional Support Conversationcited by 55
  2. 2024Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise Comparisonscited by 22
  3. 2024A Statistical Framework for Ranking LLM-Based Chatbotscited by 11
  4. 2024Finding the Subjective Truth: Collecting 2 Million Votes for Comprehensive Gen-AI Model Evaluationcited by 7
  5. 2024AutoPersuade: A Framework for Evaluating and Explaining Persuasive Argumentscited by 7
  6. 2023On Sinkhorn's Algorithm and Choice Modelingcited by 6
  7. 2023Luck, skill, and depth of competition in games and social hierarchiescited by 5
  8. 2024Statistical ranking with dynamic covariatescited by 5
  9. 2025Generalised Probabilistic Modelling and Improved Uncertainty Estimation in Comparative LLM-as-a-judgecited by 4
  10. 2025Confidence and Stability of Global and Pairwise Scores in NLP Evaluationcited by 4
  11. 2024Reliable, Reproducible, and Really Fast Leaderboards with Evalicacited by 3
  12. 2025Estimation of partial rankings from sparse, noisy comparisonscited by 3
19 more
  1. 2024Statistical inference for pairwise comparison modelscited by 2
  2. 2025C2-DPO: Constrained Controlled Direct Preference Optimizationcited by 2
  3. 2025Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modelingcited by 1
  4. 2025Interestingness First Classifierscited by 1
  5. 2024iREPO: implicit Reward Pairwise Difference based Empirical Preference Optimizationcited by 0
  6. 2024Absolute Ranking: An Essential Normalization for Benchmarking Optimization Algorithmscited by 0
  7. 2024Ranking of Multi-Response Experiment Treatmentscited by 0
  8. 2025Efficient Portfolio Selection through Preference Aggregation with Quicksort and the Bradley--Terry Modelcited by 0
  9. 2025Estimating the Error of Large Language Models at Pairwise Text Comparisoncited by 0
  10. 2026Distribution-informed Efficient Conformal Prediction for Full Rankingcited by 0
  11. 2026On rankings in multiplayer games with an application to the game of Whistcited by 0
  12. 2026New insights into Elo algorithm for practitioners and statisticianscited by 0
  13. 2026Calibrated Preference Learning: The Case of Label Rankingcited by 0
  14. 2026Pairwise Reference Alignment as a Model-Level Ordinal Observablecited by 0
  15. 2026Convergence analysis of a family of Zermelo-type iterations for the Bradley--Terry modelcited by 0
  16. 2026ToSCA: Leveraging Hierarchical Reinforcement Learning on Temporal and Strategic Abstractions of Conversational Agentscited by 0
  17. 2026CoArena: Evaluating Computer-Use and Multi-Agent Systems in Real Timecited by 0
  18. 2026Steering the Compass: Aligning Dynamic Psychological Counseling Conversations with Cognitive Behavioral Therapy Strategiescited by 0
  19. 2026Multiple latent orderings better predict language model preferencescited by 0