# Overview of the TREC 2020 Fair Ranking Track

arXiv: https://arxiv.org/abs/2108.05135 · HTML: https://mlsanity.com/paper/2108.05135/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 1 references that are in the map.
- First posted on arXiv 11 Aug 2021.

## Builds on (most cited first, 1 of 1)

- [Evaluating Stochastic Rankings with Expected Exposure](https://mlsanity.com/paper/2004.13157/) (2020)

## Built on by (most cited first, 4 of 4)

- [Introducing the Expohedron for Efficient Pareto-optimal Fairness-Utility Amortizations in Repeated Rankings](https://mlsanity.com/paper/2202.03237/) (2022)
- [The Role of Relevance in Fair Ranking](https://mlsanity.com/paper/2305.05608/) (2023)
- [Towards Efficient Pareto-optimal Utility-Fairness between Groups in Repeated Rankings](https://mlsanity.com/paper/2402.14305/) (2024)
- [The Quest for Reliable Metrics of Responsible AI](https://mlsanity.com/paper/2510.26007/) (2025)

## Abstract (arXiv)

This paper provides an overview of the NIST TREC 2020 Fair Ranking track. For 2020, we again adopted an academic search task, where we have a corpus of academic article abstracts and queries submitted to a production academic search engine. The central goal of the Fair Ranking track is to provide fair exposure to different groups of authors (a group fairness framing). We recognize that there may be multiple group definitions (e.g. based on demographics, stature, topic) and hoped for the systems to be robust to these. We expected participants to develop systems that optimize for fairness and relevance for arbitrary group definitions, and did not reveal the exact group definitions until after the evaluation runs were submitted.The track contains two tasks,reranking and retrieval, with a shared evaluation.