# Scoring Is Not Enough: Addressing Gaps in Utility-fairness Trade-offs for Ranking

arXiv: https://arxiv.org/abs/2606.26369 · HTML: https://mlsanity.com/paper/2606.26369/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Builds on (most cited first, 26 of 26)

- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [Fairness of Exposure in Rankings](https://mlsanity.com/paper/1802.07281/) (2018)
- [Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search](https://mlsanity.com/paper/1905.01989/) (2019)
- [Fairness in Recommendation Ranking through Pairwise Comparisons](https://mlsanity.com/paper/1903.00780/) (2019)
- [Equity of Attention: Amortizing Individual Fairness in Rankings](https://mlsanity.com/paper/1805.01788/) (2018)
- [Gradient Estimation Using Stochastic Computation Graphs](https://mlsanity.com/paper/1506.05254/) (2015)
- [Backpropagation through the Void: Optimizing control variates for black-box gradient estimation](https://mlsanity.com/paper/1711.00123/) (2017)
- [A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning](https://mlsanity.com/paper/2110.01515/) (2021)
- [Measuring Fairness in Ranked Outputs](https://mlsanity.com/paper/1610.08559/) (2016)
- [Policy Learning for Fairness in Ranking](https://mlsanity.com/paper/1902.04056/) (2019)
- [Designing Fair Ranking Schemes](https://mlsanity.com/paper/1712.09752/) (2017)
- [Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness](https://mlsanity.com/paper/2105.00855/) (2021)
- [Estimating Gradients for Discrete Random Variables by Sampling without Replacement](https://mlsanity.com/paper/2002.06043/) (2020)
- [Toward Pareto Efficient Fairness-Utility Trade-off inRecommendation through Reinforcement Learning](https://mlsanity.com/paper/2201.00140/) (2022)
- [CPFair: Personalized Consumer and Producer Fairness Re-ranking for Recommender Systems](https://mlsanity.com/paper/2204.08085/) (2022)
- [Fair Ranking as Fair Division: Impact-Based Individual Fairness in Ranking](https://mlsanity.com/paper/2206.07247/) (2022)
- [Low-variance Black-box Gradient Estimates for the Plackett-Luce Distribution](https://mlsanity.com/paper/1911.10036/) (2019)
- [Learning-to-Rank with Partitioned Preference: Fast Estimation for the Plackett-Luce Model](https://mlsanity.com/paper/2006.05067/) (2020)
- [Fairness for Robust Learning to Rank](https://mlsanity.com/paper/2112.06288/) (2021)
- [User-item fairness tradeoffs in recommendations](https://mlsanity.com/paper/2412.04466/) (2024)
- [Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Ranking](https://mlsanity.com/paper/2204.13765/) (2022)
- [From Query-By-Keyword to Query-By-Example: LinkedIn Talent Search Approach](https://mlsanity.com/paper/1709.00653/) (2017)
- [Algorithm-Assisted Decision Making and Racial Disparities in Housing: A Study of the Allegheny Housing Assessment Tool](https://mlsanity.com/paper/2407.21209/) (2024)
- [An External Fairness Evaluation of LinkedIn Talent Search](https://mlsanity.com/paper/2511.10752/) (2025)
- [Plackett-Luce model for learning-to-rank task](https://mlsanity.com/paper/1909.06722/) (2019)
- [Optimizing Group-Fair Plackett-Luce Ranking Models for Relevance and Ex-Post Fairness](https://mlsanity.com/paper/2308.13242/) (2023)

## Abstract (arXiv)

Scoring functions are used to represent the relevance of individual documents. In modern information retrieval or recommendation systems, they are often learned from data and play a pivotal role in ranking sets of documents or items in a way that maximizes utility to a query or user. With the recent interest in algorithmic fairness, the success of scoring has naturally led to methods that learn scores that simultaneously trade off fairness and utility. In this work, we show that in stark contrast with utility-centric objectives, scoring is sub-optimal in achieving all utility-fairness trade-offs. We establish this with a series of counter-examples with a generic fairness formulation. We show that the issue persists whether we have a deterministic scoring function or a randomized one, or whether we measure fairness at the scope of a single query or across multiple queries. On the positive side, we empirically demonstrate that semi-greedy post-processing has the potential to achieve much better trade-offs, often approaching the ideal of exhaustive post-processing in a tractable way.