mlsanityData to 7 Oct 2026

Measuring Fairness in Ranked Outputs

arXiv 1610.08559 ↗First posted 26 Oct 2016cs.DBcited by 81

Not on the map. Why?

Abstract (arXiv)

Ranking and scoring are ubiquitous. We consider the setting in which an institution, called a ranker, evaluates a set of individuals based on demographic, behavioral or other characteristics. The final output is a ranking that represents the relative quality of the individuals. While automatic and therefore seemingly objective, rankers can, and often do, discriminate against individuals and systematically disadvantage members of protected groups. This warrants a careful study of the fairness of a ranking scheme. In this paper we propose fairness measures for ranked outputs. We develop a data generation procedure that allows us to systematically control the degree of unfairness in the output, and study the behavior of our measures on these datasets. We then apply our proposed measures to several real datasets, and demonstrate cases of unfairness. Finally, we show preliminary results of incorporating our ranked fairness measures into an optimization framework, and show potential for improving fairness of ranked outputs while maintaining accuracy.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Oct 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 21; 2023: 13; 2024: 14; 2025: 12; 2026: 4.

81 papers.

Sort:

By area:

  1. 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learningcited by 104
  2. 2023VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolutioncited by 52
  3. 2021Characterizing Intersectional Group Fairness with Worst-Case Comparisonscited by 42
  4. 2021Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankerscited by 34
  5. 2023Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitmentcited by 32
  6. 2021Better Algorithms for Individually Fair k-Clusteringcited by 28
  7. 2021Fairness in Ranking under Uncertaintycited by 27
  8. 2021When Fair Ranking Meets Uncertain Inferencecited by 22
  9. 2021RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystemscited by 21
  10. 2021Maxmin-Fair Ranking: Individual Fairness under Group-Fairness Constraintscited by 18
  11. 2022Fair Ranking with Noisy Protected Attributescited by 17
  12. 2024Do Large Language Models Rank Fairly? An Empirical Study on the Fairness of LLMs as Rankerscited by 17
38 more
  1. 2024Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generationcited by 15
  2. 2021Estimation of Fair Ranking Metrics with Incomplete Judgmentscited by 14
  3. 2023A Survey on Fairness-aware Recommender Systemscited by 14
  4. 2022Fair Ranking as Fair Division: Impact-Based Individual Fairness in Rankingcited by 13
  5. 2023Invisible Relevance Bias: Text-Image Retrieval Models Prefer AI-Generated Imagescited by 13
  6. 2025Addressing Bias in LLMs: Strategies and Application to Fair AI-based Recruitmentcited by 12
  7. 2024Fairness in Algorithmic Recourse Through the Lens of Substantive Equality of Opportunitycited by 9
  8. 2021Fairness Through Regularization for Learning to Rankcited by 8
  9. 2022Measuring Disparate Outcomes of Content Recommendation Algorithms with Distributional Inequality Metricscited by 8
  10. 2021Individually Fair Rankingcited by 7
  11. 2021Identifying Biased Subgroups in Ranking and Classificationcited by 6
  12. 2021Fairness for Robust Learning to Rankcited by 6
  13. 2022Improved Approximation for Fair Correlation Clusteringcited by 6
  14. 2024Measuring and Addressing Indexical Bias in Information Retrievalcited by 6
  15. 2024Toward Structure Fairness in Dynamic Graph Embedding: A Trend-aware Dual Debiasing Approachcited by 6
  16. 202410 Years of Fair Representations: Challenges and Opportunitiescited by 6
  17. 2022Fair Group-Shared Representations with Normalizing Flowscited by 5
  18. 2023Bipartite Ranking Fairness through a Model Agnostic Ordering Adjustmentcited by 5
  19. 2025Bias-Aware Agent: Enhancing Fairness in AI-Driven Knowledge Retrievalcited by 5
  20. 2021A Ranking Approach to Fair Classificationcited by 4
  21. 2022Unintended Bias in Language Model-driven Conversational Recommendationcited by 4
  22. 2022Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Rankingcited by 4
  23. 2022Towards Fairness-Aware Multi-Objective Optimizationcited by 4
  24. 2023Fairness in Ranking under Disparate Uncertaintycited by 4
  25. 2024Measuring Bias in a Ranked List using Term-based Representationscited by 4
  26. 2024Fooling SHAP with Output Shuffling Attackscited by 4
  27. 2024Fair Secretaries with Unfair Predictionscited by 4
  28. 2021DiRe Committee : Diversity and Representation Constraints in Multiwinner Electionscited by 3
  29. 2022Fair Interpretable Representation Learning with Correction Vectorscited by 3
  30. 2021FaiREO: User Group Fairness for Equality of Opportunity in Course Recommendationcited by 2
  31. 2022MANI-Rank: Multiple Attribute and Intersectional Group Fairness for Consensus Rankingcited by 2
  32. 2022Contextual bandits with concave rewards, and an application to fair rankingcited by 2
  33. 2022Properties of Group Fairness Metrics for Rankingscited by 2
  34. 2023Biases in Scholarly Recommender Systems: Impact, Prevalence, and Mitigationcited by 2
  35. 2023On the Within-Group Fairness of Screening Classifierscited by 2
  36. 2023Sampling Individually-Fair Rankings that are Always Group Faircited by 2
  37. 2024Perceptual Fairness in Image Restorationcited by 2
  38. 2025Efficient and Responsible Adaptation of Large Language Models for Robust and Equitable Top-k Recommendationscited by 2

50 most cited of 81 All 81 (JSON)