# Metrics for popularity bias in dynamic recommender systems

arXiv: https://arxiv.org/abs/2310.08455 · HTML: https://mlsanity.com/paper/2310.08455/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 73 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 18 of 18)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Bias and Debias in Recommender System: A Survey and Future Directions](https://mlsanity.com/paper/2010.03240/) (2020)
- [A Survey on the Fairness of Recommender Systems](https://mlsanity.com/paper/2206.03761/) (2022)
- [A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices](https://mlsanity.com/paper/1807.00787/) (2018)
- [Feedback Loop and Bias Amplification in Recommender Systems](https://mlsanity.com/paper/2007.13019/) (2020)
- [The Unfairness of Popularity Bias in Recommendation](https://mlsanity.com/paper/1907.13286/) (2019)
- [User-centered Evaluation of Popularity Bias in Recommender Systems](https://mlsanity.com/paper/2103.06364/) (2021)
- [Fairness in Recommender Systems: Research Landscape and Future Directions](https://mlsanity.com/paper/2205.11127/) (2022)
- [The Unfairness of Popularity Bias in Music Recommendation: A Reproducibility Study](https://mlsanity.com/paper/1912.04696/) (2019)
- [Multi-sided Exposure Bias in Recommendation](https://mlsanity.com/paper/2006.15772/) (2020)
- [Popularity Bias Is Not Always Evil: Disentangling Benign and Harmful Bias for Recommendation](https://mlsanity.com/paper/2109.07946/) (2021)
- [User Fairness in Recommender Systems](https://mlsanity.com/paper/1807.06349/) (2018)
- [A Study of Position Bias in Digital Library Recommender Systems](https://mlsanity.com/paper/1802.06565/) (2018)
- [Quantifying and Mitigating Popularity Bias in Conversational Recommender Systems](https://mlsanity.com/paper/2208.03298/) (2022)
- [Popularity Bias in Collaborative Filtering-Based Multimedia Recommender Systems](https://mlsanity.com/paper/2203.00376/) (2022)
- [Theoretical Modeling of the Iterative Properties of User Discovery in a Collaborative Filtering Recommender System](https://mlsanity.com/paper/2008.13526/) (2020)
- [Use of Deep Learning in Modern Recommendation System: A Summary of Recent Works](https://mlsanity.com/paper/1712.07525/) (2017)
- [Quantifying the Effects of Recommendation Systems](https://mlsanity.com/paper/2002.01077/) (2020)

## Built on by (most cited first, 3 of 3)

- [Adaptive Quality-Diversity Trade-offs for Large-Scale Batch Recommendation](https://mlsanity.com/paper/2602.02024/) (2026)
- [Principled Synthetic Data Enables the First Scaling Laws for LLMs in Recommendation](https://mlsanity.com/paper/2602.07298/) (2026)
- [Aligning Recommendations with User Popularity Preferences](https://mlsanity.com/paper/2604.01036/) (2026)

## Abstract (arXiv)

Albeit the widespread application of recommender systems (RecSys) in our daily lives, rather limited research has been done on quantifying unfairness and biases present in such systems. Prior work largely focuses on determining whether a RecSys is discriminating or not but does not compute the amount of bias present in these systems. Biased recommendations may lead to decisions that can potentially have adverse effects on individuals, sensitive user groups, and society. Hence, it is important to quantify these biases for fair and safe commercial applications of these systems. This paper focuses on quantifying popularity bias that stems directly from the output of RecSys models, leading to over recommendation of popular items that are likely to be misaligned with user preferences. Four metrics to quantify popularity bias in RescSys over time in dynamic setting across different sensitive user groups have been proposed. These metrics have been demonstrated for four collaborative filtering based RecSys algorithms trained on two commonly used benchmark datasets in the literature. Results obtained show that the metrics proposed provide a comprehensive understanding of growing disparities in treatment between sensitive groups over time when used conjointly.