# A Study on Accuracy, Miscalibration, and Popularity Bias in Recommendations

arXiv: https://arxiv.org/abs/2303.00400 · HTML: https://mlsanity.com/paper/2303.00400/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 33 references that are in the map.
- First posted on arXiv 1 Mar 2023.

## Builds on (most cited first, 10 of 10)

- [Managing Popularity Bias in Recommender Systems with Personalized Re-ranking](https://mlsanity.com/paper/1901.07555/) (2019)
- [The Unfairness of Popularity Bias in Recommendation](https://mlsanity.com/paper/1907.13286/) (2019)
- [Sequence-Aware Recommender Systems](https://mlsanity.com/paper/1802.08452/) (2018)
- [User-centered Evaluation of Popularity Bias in Recommender Systems](https://mlsanity.com/paper/2103.06364/) (2021)
- [The Connection Between Popularity Bias, Calibration, and Fairness in Recommendation](https://mlsanity.com/paper/2008.09273/) (2020)
- [The Unfairness of Popularity Bias in Music Recommendation: A Reproducibility Study](https://mlsanity.com/paper/1912.04696/) (2019)
- [The Impact of Popularity Bias on Fairness and Calibration in Recommendation](https://mlsanity.com/paper/1910.05755/) (2019)
- [Analyzing Item Popularity Bias of Music Recommender Systems: Are Different Genders Equally Affected?](https://mlsanity.com/paper/2108.06973/) (2021)
- [Popularity Bias in Collaborative Filtering-Based Multimedia Recommender Systems](https://mlsanity.com/paper/2203.00376/) (2022)
- [Support the Underground: Characteristics of Beyond-Mainstream Music Listeners](https://mlsanity.com/paper/2102.12188/) (2021)

## Abstract (arXiv)

Recent research has suggested different metrics to measure the inconsistency of recommendation performance, including the accuracy difference between user groups, miscalibration, and popularity lift. However, a study that relates miscalibration and popularity lift to recommendation accuracy across different user groups is still missing. Additionally, it is unclear if particular genres contribute to the emergence of inconsistency in recommendation performance across user groups. In this paper, we present an analysis of these three aspects of five well-known recommendation algorithms for user groups that differ in their preference for popular content. Additionally, we study how different genres affect the inconsistency of recommendation performance, and how this is aligned with the popularity of the genres. Using data from LastFm, MovieLens, and MyAnimeList, we present two key findings. First, we find that users with little interest in popular content receive the worst recommendation accuracy, and that this is aligned with miscalibration and popularity lift. Second, our experiments show that particular genres contribute to a different extent to the inconsistency of recommendation performance, especially in terms of miscalibration in the case of the MyAnimeList dataset.