# Online certification of preference-based fairness for personalized recommender systems

arXiv: https://arxiv.org/abs/2104.14527 · HTML: https://mlsanity.com/paper/2104.14527/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 70 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 27 of 27)

- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Fairness of Exposure in Rankings](https://mlsanity.com/paper/1802.07281/) (2018)
- [Fairness in Learning: Classic and Contextual Bandits](https://mlsanity.com/paper/1605.07139/) (2016)
- [lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Bandits](https://mlsanity.com/paper/1312.7308/) (2013)
- [Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search](https://mlsanity.com/paper/1905.01989/) (2019)
- [Equity of Attention: Amortizing Individual Fairness in Rankings](https://mlsanity.com/paper/1805.01788/) (2018)
- [Fairness in Reinforcement Learning](https://mlsanity.com/paper/1611.03071/) (2016)
- [Discrimination in Online Ad Delivery](https://mlsanity.com/paper/1301.6822/) (2013)
- [Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination](https://mlsanity.com/paper/1408.6491/) (2014)
- [Ranking with Fairness Constraints](https://mlsanity.com/paper/1704.06840/) (2017)
- [FairRec: Two-Sided Fairness for Personalized Recommendations in Two-Sided Platforms](https://mlsanity.com/paper/2002.10764/) (2020)
- [Translation-based Recommendation](https://mlsanity.com/paper/1707.02410/) (2017)
- [Multisided Fairness for Recommendation](https://mlsanity.com/paper/1707.00093/) (2017)
- [An optimal algorithm for the Thresholding Bandit Problem](https://mlsanity.com/paper/1605.08671/) (2016)
- [From Parity to Preference-based Notions of Fairness in Classification](https://mlsanity.com/paper/1707.00010/) (2017)
- [Conservative Bandits](https://mlsanity.com/paper/1602.04282/) (2016)
- [Calibrated Fairness in Bandits](https://mlsanity.com/paper/1707.01875/) (2017)
- [Good Arm Identification via Bandit Feedback](https://mlsanity.com/paper/1710.06360/) (2017)
- [Conservative Exploration in Reinforcement Learning](https://mlsanity.com/paper/2002.03218/) (2020)
- [Auditing for Discrimination in Algorithms Delivering Job Ads](https://mlsanity.com/paper/2104.04502/) (2021)
- [Improved Algorithms for Conservative Exploration in Bandits](https://mlsanity.com/paper/2002.03221/) (2020)
- [Preference-Informed Fairness](https://mlsanity.com/paper/1904.01793/) (2019)
- [Auditing Search Engines for Differential Satisfaction Across Demographics](https://mlsanity.com/paper/1705.10689/) (2017)
- [Envy-Free Classification](https://mlsanity.com/paper/1809.08700/) (2018)
- [Asymptotic Existence of Fair Divisions for Groups](https://mlsanity.com/paper/1706.08219/) (2017)

## Built on by (most cited first, 22 of 22)

- [Modeling Content Creator Incentives on Algorithm-Curated Platforms](https://mlsanity.com/paper/2206.13102/) (2022)
- [Two-sided fairness in rankings via Lorenz dominance](https://mlsanity.com/paper/2110.15781/) (2021)
- [Fairness in Recommendation: Foundations, Methods and Applications](https://mlsanity.com/paper/2205.13619/) (2022)
- [A Survey on Fairness-aware Recommender Systems](https://mlsanity.com/paper/2306.00403/) (2023)
- [Recent Developments in Recommender Systems: A Survey](https://mlsanity.com/paper/2306.12680/) (2023)
- [Optimizing generalized Gini indices for fairness in rankings](https://mlsanity.com/paper/2204.06521/) (2022)
- [Fair Ranking as Fair Division: Impact-Based Individual Fairness in Ranking](https://mlsanity.com/paper/2206.07247/) (2022)
- [Introducing the Expohedron for Efficient Pareto-optimal Fairness-Utility Amortizations in Repeated Rankings](https://mlsanity.com/paper/2202.03237/) (2022)
- [Counterfactually Fair Representation](https://mlsanity.com/paper/2311.05420/) (2023)
- [Intersectional Two-sided Fairness in Recommendation](https://mlsanity.com/paper/2402.02816/) (2024)
- [When Personalization Harms: Reconsidering the Use of Group Attributes in Prediction](https://mlsanity.com/paper/2206.02058/) (2022)
- [Participatory Personalization in Classification](https://mlsanity.com/paper/2302.03874/) (2023)
- [Towards Fairness-Aware Multi-Objective Optimization](https://mlsanity.com/paper/2207.12138/) (2022)
- [Unmasking Gender Bias in Recommendation Systems and Enhancing Category-Aware Fairness](https://mlsanity.com/paper/2502.17921/) (2025)
- [Lookahead Counterfactual Fairness](https://mlsanity.com/paper/2412.01065/) (2024)
- [PA2D-MORL: Pareto Ascent Directional Decomposition based Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2603.19579/) (2026)
- [When Collaborative Filtering is not Collaborative: Unfairness of PCA for Recommendations](https://mlsanity.com/paper/2310.09687/) (2023)
- [Envy-Free but Still Unfair: Envy-Freeness Up To One Item (EF-1) in Personalized Recommendation](https://mlsanity.com/paper/2509.09037/) (2025)
- [Measuring Individual User Fairness with User Similarity and Effectiveness Disparity](https://mlsanity.com/paper/2602.02516/) (2026)
- [FEIR: Quantifying and Reducing Envy and Inferiority for Fair Recommendation of Limited Resources](https://mlsanity.com/paper/2311.04542/) (2023)
- [Towards Efficient Pareto-optimal Utility-Fairness between Groups in Repeated Rankings](https://mlsanity.com/paper/2402.14305/) (2024)
- [A Counterfactual Approach for Addressing Individual User Unfairness in Collaborative Recommender System](https://mlsanity.com/paper/2603.13253/) (2026)

## Abstract (arXiv)

Recommender systems are facing scrutiny because of their growing impact on the opportunities we have access to. Current audits for fairness are limited to coarse-grained parity assessments at the level of sensitive groups. We propose to audit for envy-freeness, a more granular criterion aligned with individual preferences: every user should prefer their recommendations to those of other users. Since auditing for envy requires to estimate the preferences of users beyond their existing recommendations, we cast the audit as a new pure exploration problem in multi-armed bandits. We propose a sample-efficient algorithm with theoretical guarantees that it does not deteriorate user experience. We also study the trade-offs achieved on real-world recommendation datasets.