# Bipartite Ranking Fairness through a Model Agnostic Ordering Adjustment

arXiv: https://arxiv.org/abs/2307.14668 · HTML: https://mlsanity.com/paper/2307.14668/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 64 references that are in the map.
- First posted on arXiv 27 Jul 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [The Variational Fair Autoencoder](https://mlsanity.com/paper/1511.00830/) (2015)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [Fairness of Exposure in Rankings](https://mlsanity.com/paper/1802.07281/) (2018)
- [Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations](https://mlsanity.com/paper/1707.00075/) (2017)
- [Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search](https://mlsanity.com/paper/1905.01989/) (2019)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [Fairness in Recommendation Ranking through Pairwise Comparisons](https://mlsanity.com/paper/1903.00780/) (2019)
- [FA*IR: A Fair Top-k Ranking Algorithm](https://mlsanity.com/paper/1706.06368/) (2017)
- [Measuring Fairness in Ranked Outputs](https://mlsanity.com/paper/1610.08559/) (2016)
- [Ranking with Fairness Constraints](https://mlsanity.com/paper/1704.06840/) (2017)
- [Policy Learning for Fairness in Ranking](https://mlsanity.com/paper/1902.04056/) (2019)
- [Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements](https://mlsanity.com/paper/1901.04562/) (2019)
- [Pairwise Fairness for Ranking and Regression](https://mlsanity.com/paper/1906.05330/) (2019)
- [Two-Player Games for Efficient Non-Convex Constrained Optimization](https://mlsanity.com/paper/1804.06500/) (2018)
- [Residual Unfairness in Fair Machine Learning from Prejudiced Data](https://mlsanity.com/paper/1806.02887/) (2018)
- [The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric](https://mlsanity.com/paper/1902.05826/) (2019)
- [Fairness in Semi-supervised Learning: Unlabeled Data Help to Reduce Discrimination](https://mlsanity.com/paper/2009.12040/) (2020)
- [Towards Model-Agnostic Post-Hoc Adjustment for Balancing Ranking Fairness and Algorithm Utility](https://mlsanity.com/paper/2006.08267/) (2020)
- [Learning Fair Scoring Functions: Bipartite Ranking under ROC-based Fairness Constraints](https://mlsanity.com/paper/2002.08159/) (2020)

## Built on by (most cited first, 5 of 5)

- [Unmasking Bias in AI: A Systematic Review of Bias Detection and Mitigation Strategies in Electronic Health Record-based Models](https://mlsanity.com/paper/2310.19917/) (2023)
- [Multi-Output Distributional Fairness via Post-Processing](https://mlsanity.com/paper/2409.00553/) (2024)
- [Critical Appraisal of Fairness Metrics in Clinical Predictive AI](https://mlsanity.com/paper/2506.17035/) (2025)
- [PAT: Pixel-wise Adaptive Training for Long-tailed Segmentation](https://mlsanity.com/paper/2404.05393/) (2024)
- [FairPOT: Balancing AUC Performance and Fairness with Proportional Optimal Transport](https://mlsanity.com/paper/2508.03940/) (2025)

## Abstract (arXiv)

Algorithmic fairness has been a serious concern and received lots of interest in machine learning community. In this paper, we focus on the bipartite ranking scenario, where the instances come from either the positive or negative class and the goal is to learn a ranking function that ranks positive instances higher than negative ones. While there could be a trade-off between fairness and performance, we propose a model agnostic post-processing framework xOrder for achieving fairness in bipartite ranking and maintaining the algorithm classification performance. In particular, we optimize a weighted sum of the utility as identifying an optimal warping path across different protected groups and solve it through a dynamic programming process. xOrder is compatible with various classification models and ranking fairness metrics, including supervised and unsupervised fairness metrics. In addition to binary groups, xOrder can be applied to multiple protected groups. We evaluate our proposed algorithm on four benchmark data sets and two real-world patient electronic health record repositories. xOrder consistently achieves a better balance between the algorithm utility and ranking fairness on a variety of datasets with different metrics. From the visualization of the calibrated ranking scores, xOrder mitigates the score distribution shifts of different groups compared with baselines. Moreover, additional analytical results verify that xOrder achieves a robust performance when faced with fewer samples and a bigger difference between training and testing ranking score distributions.