mlsanityData to 7 Oct 2026

Uncertainty Quantification for Fairness in Two-Stage Recommender Systems

arXiv 2205.15436 ↗First posted 30 May 2022cs.IR cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Many large-scale recommender systems consist of two stages. The first stage efficiently screens the complete pool of items for a small subset of promising candidates, from which the second-stage model curates the final recommendations. In this paper, we investigate how to ensure group fairness to the items in this two-stage architecture. In particular, we find that existing first-stage recommenders might select an irrecoverably unfair set of candidates such that there is no hope for the second-stage recommender to deliver fair recommendations. To this end, motivated by recent advances in uncertainty quantification, we propose two threshold-policy selection rules that can provide distribution-free and finite-sample guarantees on fairness in first-stage recommenders. More concretely, given any relevance model of queries and items and a point-wise lower confidence bound on the expected number of relevant items for each threshold-policy, the two rules find near-optimal sets of candidates that contain enough relevant items in expectation from each group of items. To instantiate the rules, we demonstrate how to derive such confidence bounds from potentially partial and biased user feedback data, which are abundant in many large-scale recommender systems. In addition, we provide both finite-sample and asymptotic analyses of how close the two threshold selection rules are to the optimal thresholds. Beyond this theoretical analysis, we show empirically that these two rules can consistently select enough relevant items from each group while minimizing the size of the candidate sets for a wide range of settings.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 43 of its 81 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

43 of 81 references.

References · 43

  1. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  2. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  3. 2017Algorithmic decision making and the cost of fairnesscited by 402
  4. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 278
  5. 2016Recommendations as Treatments: Debiasing Learning and Evaluationcited by 246
  6. 2018Fairness of Exposure in Rankingscited by 189
  7. 2018Top-K Off-Policy Correction for a REINFORCE Recommender Systemcited by 181
  8. 2016Fairness in Learning: Classic and Contextual Banditscited by 167
  9. 2016Unbiased Learning-to-Rank with Biased Feedbackcited by 155
  10. 2019Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Searchcited by 137
  11. 2019Fairness in Recommendation Ranking through Pairwise Comparisonscited by 127
  12. 2018Equity of Attention: Amortizing Individual Fairness in Rankingscited by 124
31 more
  1. 2013Introducing LETOR 4.0 Datasetscited by 120
  2. 2017FA*IR: A Fair Top-k Ranking Algorithmcited by 113
  3. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
  4. 2018Learning Tree-based Deep Model for Recommender Systemscited by 99
  5. 2017Beyond Parity: Fairness Objectives for Collaborative Filteringcited by 93
  6. 2020Distribution-free binary classification: prediction sets, confidence intervals and calibrationcited by 79
  7. 2017Ranking with Fairness Constraintscited by 78
  8. 2019Achieving Fairness in the Stochastic Multi-armed Bandit Problemcited by 77
  9. 2021Fairness in Rankings and Recommendations: An Overviewcited by 62
  10. 2018Online Learning with an Unknown Fairness Metriccited by 61
  11. 2018Unbiased Learning to Rank with Unbiased Propensity Estimationcited by 61
  12. 2020Evaluating Stochastic Rankings with Expected Exposurecited by 53
  13. 2017Balanced Policy Evaluation and Learningcited by 51
  14. 2017Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Timecited by 46
  15. 2021Fairness of Exposure in Stochastic Banditscited by 42
  16. 2019Fair Contextual Multi-Armed Bandits: Theory and Experimentscited by 41
  17. 2020Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weightingcited by 38
  18. 2018Fighting Fire with Fire: Using Antidote Data to Improve Polarization and Fairness of Recommender Systemscited by 37
  19. 2018Estimating Position Bias without Intrusive Interventionscited by 31
  20. 2021TFROM: A Two-sided Fairness-Aware Recommendation Model for Both Customers and Providerscited by 31
  21. 2021Two-sided fairness in rankings via Lorenz dominancecited by 28
  22. 2020User Fairness, Item Fairness, and Diversity for Rankings in Two-Sided Marketscited by 23
  23. 2019Candidate Generation with Binary Codes for Large-Scale Top-N Recommendationcited by 19
  24. 2022Improving Screening Processes via Calibrated Subset Selectioncited by 16
  25. 2020Cascade Model-based Propensity Estimation for Counterfactual Learning to Rankcited by 15
  26. 2021Obtaining Calibrated Probabilities with Personalized Ranking Modelscited by 15
  27. 2018Intervention Harvesting for Context-Dependent Examination-Bias Estimationcited by 14
  28. 2021On component interactions in two-stage recommender systemscited by 14
  29. 2019Group Fairness in Bandit Arm Selectioncited by 12
  30. 2022Introducing the Expohedron for Efficient Pareto-optimal Fairness-Utility Amortizations in Repeated Rankingscited by 11
  31. 2022Fairness of Exposure in Light of Incomplete Exposure Estimationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 2; 2025: 3; 2026: 1.

10 papers.

  1. 2023Unraveling Privacy Risks of Individual Fairness in Graph Neural Networkscited by 16
  2. 2025Uncertainty Quantification and Decomposition for LLM-based Recommendationcited by 12
  3. 2023CompanyKG: A Large-Scale Heterogeneous Graph for Company Similarity Quantificationcited by 6
  4. 2024Two-stage Risk Control with Application to Ranked Retrievalcited by 6
  5. 2023When Are Two Lists Better than One?: Benefits and Harms in Joint Decision-makingcited by 4
  6. 2023Fairness in Ranking under Disparate Uncertaintycited by 4
  7. 2025You Don't Bring Me Flowers: Mitigating Unwanted Recommendations Through Conformal Risk Controlcited by 3
  8. 2024Calibration-Disentangled Learning and Relevance-Prioritized Reranking for Calibrated Sequential Recommendationcited by 1
  9. 2026User-Adaptive Meta-Learning for Cold-Start Medication Recommendation with Uncertainty Filteringcited by 1
  10. 2025Session-based Recommender Systems: User Interest as a Stochastic Process in the Latent Spacecited by 0