# Ranking a Set of Objects using Heterogeneous Workers: QUITE an Easy Problem

arXiv: https://arxiv.org/abs/2310.02016 · HTML: https://mlsanity.com/paper/2310.02016/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 3 Oct 2023.

## Builds on (most cited first, 11 of 11)

- [Simple, Robust and Optimal Ranking from Pairwise Comparisons](https://mlsanity.com/paper/1512.08949/) (2015)
- [Lagrangian Inference for Ranking Problems](https://mlsanity.com/paper/2110.00151/) (2021)
- [Maximum Selection and Ranking under Noisy Comparisons](https://mlsanity.com/paper/1705.05366/) (2017)
- [Rank Aggregation via Heterogeneous Thurstone Preference Models](https://mlsanity.com/paper/1912.01211/) (2019)
- [Top-K Ranking from Pairwise Comparisons: When Spectral Ranking is Optimal](https://mlsanity.com/paper/1603.04153/) (2016)
- [Ranking and synchronization from pairwise measurements via SVD](https://mlsanity.com/paper/1906.02746/) (2019)
- [ℓ∞-Bounds of the MLE in the BTL Model under General Comparison Graphs](https://mlsanity.com/paper/2110.10825/) (2021)
- [Graph Resistance and Learning from Pairwise Comparisons](https://mlsanity.com/paper/1902.00141/) (2019)
- [Bayesian Decision Process for Cost-Efficient Dynamic Ranking via Crowdsourcing](https://mlsanity.com/paper/1612.07222/) (2016)
- [Adaptive Sampling for Heterogeneous Rank Aggregation from Noisy Pairwise Comparisons](https://mlsanity.com/paper/2110.04136/) (2021)
- [Ranking a set of objects: a graph based least-square approach](https://mlsanity.com/paper/2002.11590/) (2020)

## Abstract (arXiv)

We focus on the problem of ranking N objects starting from a set of noisy pairwise comparisons provided by a crowd of unequal workers, each worker being characterized by a specific degree of reliability, which reflects her ability to rank pairs of objects. More specifically, we assume that objects are endowed with intrinsic qualities and that the probability with which an object is preferred to another depends both on the difference between the qualities of the two competitors and on the reliability of the worker. We propose QUITE, a non-adaptive ranking algorithm that jointly estimates workers' reliabilities and qualities of objects. Performance of QUITE is compared in different scenarios against previously proposed algorithms. Finally, we show how QUITE can be naturally made adaptive.