# Quantifying the Impact of User Attention on Fair Group Representation in Ranked Lists

arXiv: https://arxiv.org/abs/1901.10437 · HTML: https://mlsanity.com/paper/1901.10437/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2019.

## Built on by (most cited first, 24 of 24)

- [Characterizing Intersectional Group Fairness with Worst-Case Comparisons](https://mlsanity.com/paper/2101.01673/) (2021)
- [When Fair Ranking Meets Uncertain Inference](https://mlsanity.com/paper/2105.02091/) (2021)
- [Fair ranking: a critical review, challenges, and future directions](https://mlsanity.com/paper/2201.12662/) (2022)
- [Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation](https://mlsanity.com/paper/2409.11598/) (2024)
- [Invisible Relevance Bias: Text-Image Retrieval Models Prefer AI-Generated Images](https://mlsanity.com/paper/2311.14084/) (2023)
- [Fairness Through Regularization for Learning to Rank](https://mlsanity.com/paper/2102.05996/) (2021)
- [Measuring Disparate Outcomes of Content Recommendation Algorithms with Distributional Inequality Metrics](https://mlsanity.com/paper/2202.01615/) (2022)
- [Individually Fair Ranking](https://mlsanity.com/paper/2103.11023/) (2021)
- [Fairness for Robust Learning to Rank](https://mlsanity.com/paper/2112.06288/) (2021)
- [Measuring and Addressing Indexical Bias in Information Retrieval](https://mlsanity.com/paper/2406.04298/) (2024)
- [Subverting Fair Image Search with Generative Adversarial Perturbations](https://mlsanity.com/paper/2205.02414/) (2022)
- [Language Fairness in Multilingual Information Retrieval](https://mlsanity.com/paper/2405.00978/) (2024)
- [Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Ranking](https://mlsanity.com/paper/2204.13765/) (2022)
- [Fairness in Ranking under Disparate Uncertainty](https://mlsanity.com/paper/2309.01610/) (2023)
- [Measuring Bias in a Ranked List using Term-based Representations](https://mlsanity.com/paper/2403.05975/) (2024)
- [The Role of Relevance in Fair Ranking](https://mlsanity.com/paper/2305.05608/) (2023)
- [Unmasking Gender Bias in Recommendation Systems and Enhancing Category-Aware Fairness](https://mlsanity.com/paper/2502.17921/) (2025)
- [Language Bias in Information Retrieval: The Nature of the Beast and Mitigation Methods](https://mlsanity.com/paper/2509.06195/) (2025)
- [Properties of Group Fairness Metrics for Rankings](https://mlsanity.com/paper/2212.14351/) (2022)
- [Does Reasoning Make Search More Fair? Comparing Fairness in Reasoning and Non-Reasoning Rerankers](https://mlsanity.com/paper/2603.10332/) (2026)
- [A Reproducibility Study of Product-side Fairness in Bundle Recommendation](https://mlsanity.com/paper/2507.14352/) (2025)
- [Navigating the Thin Line: Examining User Behavior in Search to Detect Engagement and Backfire Effects](https://mlsanity.com/paper/2401.11201/) (2024)
- [Towards Fair Rankings: Leveraging LLMs for Gender Bias Detection and Measurement](https://mlsanity.com/paper/2506.22372/) (2025)
- [Fairness in Link Prediction Beyond Demographic Parity: A Reproducibility Study](https://mlsanity.com/paper/2608.09899/) (2026)

## Abstract (arXiv)

In this work, we introduce a novel metric for auditing group fairness in ranked lists. Our approach offers two benefits compared to the state of the art. First, we offer a blueprint for modeling of user attention. Rather than assuming a logarithmic loss in importance as a function of the rank, we can account for varying user behaviors through parametrization. For example, we expect a user to see more items during a viewing of a social media feed than when they inspect the results list of a single web search query. Second, we allow non-binary protected attributes to enable investigating inherently continuous attributes (eg political alignment on the liberal to conservative spectrum) as well as to facilitate measurements across aggregated sets of search results, rather than separately for each result list. By combining these two elements into our metric, we are able to better address the human factors inherent in this problem. We measure the whole sociotechnical system, consisting of a ranking algorithm and individuals using it, instead of exclusively focusing on the ranking algorithm. Finally, we use our metric to perform three simulated fairness audits. We show that determining fairness of a ranked output necessitates knowledge (or a model) of the end-users of the particular service. Depending on their attention distribution function, a fixed ranking of results can appear biased both in favor and against a protected group.