# Interventions for Ranking in the Presence of Implicit Bias

arXiv: https://arxiv.org/abs/2001.08767 · HTML: https://mlsanity.com/paper/2001.08767/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2020.

## Built on by (most cited first, 27 of 27)

- [Fairness in Ranking under Uncertainty](https://mlsanity.com/paper/2107.06720/) (2021)
- [A Sociotechnical View of Algorithmic Fairness](https://mlsanity.com/paper/2110.09253/) (2021)
- [Maxmin-Fair Ranking: Individual Fairness under Group-Fairness Constraints](https://mlsanity.com/paper/2106.08652/) (2021)
- [Fair Ranking with Noisy Protected Attributes](https://mlsanity.com/paper/2211.17067/) (2022)
- [Fair Ranking as Fair Division: Impact-Based Individual Fairness in Ranking](https://mlsanity.com/paper/2206.07247/) (2022)
- [On Fair Selection in the Presence of Implicit and Differential Variance](https://mlsanity.com/paper/2112.05630/) (2021)
- [Fairness Through Regularization for Learning to Rank](https://mlsanity.com/paper/2102.05996/) (2021)
- [Individually Fair Ranking](https://mlsanity.com/paper/2103.11023/) (2021)
- [Fairness for Robust Learning to Rank](https://mlsanity.com/paper/2112.06288/) (2021)
- [Selection in the Presence of Implicit Bias: The Advantage of Intersectional Constraints](https://mlsanity.com/paper/2202.01661/) (2022)
- [Fairness in Matching under Uncertainty](https://mlsanity.com/paper/2302.03810/) (2023)
- [Maximizing Submodular Functions for Recommendation in the Presence of Biases](https://mlsanity.com/paper/2305.02806/) (2023)
- [Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Ranking](https://mlsanity.com/paper/2204.13765/) (2022)
- [Fairness in Ranking under Disparate Uncertainty](https://mlsanity.com/paper/2309.01610/) (2023)
- [On Meritocracy in Optimal Set Selection](https://mlsanity.com/paper/2102.11932/) (2021)
- [The Equity Framework: Fairness Beyond Equalized Predictive Outcomes](https://mlsanity.com/paper/2205.01072/) (2022)
- [Properties of Group Fairness Metrics for Rankings](https://mlsanity.com/paper/2212.14351/) (2022)
- [Subset Selection Based On Multiple Rankings in the Presence of Bias: Effectiveness of Fairness Constraints for Multiwinner Voting Score Functions](https://mlsanity.com/paper/2306.09835/) (2023)
- [A Mathematical Framework for AI-Human Integration in Work](https://mlsanity.com/paper/2505.23432/) (2025)
- [Optimizing Group-Fair Plackett-Luce Ranking Models for Relevance and Ex-Post Fairness](https://mlsanity.com/paper/2308.13242/) (2023)
- [Learning Fair Policies for Multi-stage Selection Problems from Observational Data](https://mlsanity.com/paper/2312.13173/) (2023)
- [Sampling Ex-Post Group-Fair Rankings](https://mlsanity.com/paper/2203.00887/) (2022)
- [TAPS Responsibility Matrix: A tool for responsible data science by design](https://mlsanity.com/paper/2302.01041/) (2023)
- [Fair Active Ranking from Pairwise Preferences](https://mlsanity.com/paper/2402.03252/) (2024)
- [Centralized Selection with Preferences in the Presence of Biases](https://mlsanity.com/paper/2409.04897/) (2024)
- [Reducing Biases in Record Matching Through Scores Calibration](https://mlsanity.com/paper/2411.01685/) (2024)
- [A Tutorial On Intersectionality in Fair Rankings](https://mlsanity.com/paper/2502.05333/) (2025)

## Abstract (arXiv)

Implicit bias is the unconscious attribution of particular qualities (or lack thereof) to a member from a particular social group (e.g., defined by gender or race). Studies on implicit bias have shown that these unconscious stereotypes can have adverse outcomes in various social contexts, such as job screening, teaching, or policing. Recently, (Kleinberg and Raghavan, 2018) considered a mathematical model for implicit bias and showed the effectiveness of the Rooney Rule as a constraint to improve the utility of the outcome for certain cases of the subset selection problem. Here we study the problem of designing interventions for the generalization of subset selection -- ranking -- that requires to output an ordered set and is a central primitive in various social and computational contexts. We present a family of simple and interpretable constraints and show that they can optimally mitigate implicit bias for a generalization of the model studied in (Kleinberg and Raghavan, 2018). Subsequently, we prove that under natural distributional assumptions on the utilities of items, simple, Rooney Rule-like, constraints can also surprisingly recover almost all the utility lost due to implicit biases. Finally, we augment our theoretical results with empirical findings on real-world distributions from the IIT-JEE (2009) dataset and the Semantic Scholar Research corpus.