# Break Out of a Pigeonhole: A Unified Framework for Examining Miscalibration, Bias, and Stereotype in Recommender Systems

arXiv: https://arxiv.org/abs/2312.17443 · HTML: https://mlsanity.com/paper/2312.17443/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 56 references that are in the map.
- First posted on arXiv 29 Dec 2023.

## Builds on (most cited first, 22 of 22)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](https://mlsanity.com/paper/1710.11214/) (2017)
- [FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning](https://mlsanity.com/paper/1904.05419/) (2019)
- [FairSight: Visual Analytics for Fairness in Decision Making](https://mlsanity.com/paper/1908.00176/) (2019)
- [Deconfounded Recommendation for Alleviating Bias Amplification](https://mlsanity.com/paper/2105.10648/) (2021)
- [Understanding Echo Chambers in E-commerce Recommender Systems](https://mlsanity.com/paper/2007.02474/) (2020)
- [The Connection Between Popularity Bias, Calibration, and Fairness in Recommendation](https://mlsanity.com/paper/2008.09273/) (2020)
- [Socially-Aware Self-Supervised Tri-Training for Recommendation](https://mlsanity.com/paper/2106.03569/) (2021)
- [Exploring Author Gender in Book Rating and Recommendation](https://mlsanity.com/paper/1808.07586/) (2018)
- [Fairness in representation: quantifying stereotyping as a representational harm](https://mlsanity.com/paper/1901.09565/) (2019)
- [User-controllable Recommendation Against Filter Bubbles](https://mlsanity.com/paper/2204.13844/) (2022)
- [Bias Disparity in Recommendation Systems](https://mlsanity.com/paper/1811.01461/) (2018)
- [Crank up the volume: preference bias amplification in collaborative recommendation](https://mlsanity.com/paper/1909.06362/) (2019)
- [Towards Involving End-users in Interactive Human-in-the-loop AI Fairness](https://mlsanity.com/paper/2204.10464/) (2022)
- [The Stereotyping Problem in Collaboratively Filtered Recommender Systems](https://mlsanity.com/paper/2106.12622/) (2021)
- [The Impact of Popularity Bias on Fairness and Calibration in Recommendation](https://mlsanity.com/paper/1910.05755/) (2019)
- [Gender Stereotype Reinforcement: Measuring the Gender Bias Conveyed by Ranking Algorithms](https://mlsanity.com/paper/2009.01334/) (2020)
- [Towards Responsible AI: A Design Space Exploration of Human-Centered Artificial Intelligence User Interfaces to Investigate Fairness](https://mlsanity.com/paper/2206.00474/) (2022)
- [The Minority Matters: A Diversity-Promoting Collaborative Metric Learning Algorithm](https://mlsanity.com/paper/2209.15292/) (2022)
- [Bias Disparity in Collaborative Recommendation: Algorithmic Evaluation and Comparison](https://mlsanity.com/paper/1908.00831/) (2019)
- [Investigating Potential Factors Associated with Gender Discrimination in Collaborative Recommender Systems](https://mlsanity.com/paper/2002.07786/) (2020)

## Built on by (most cited first, 2 of 2)

- [Interactive Counterfactual Exploration of Algorithmic Harms in Recommender Systems](https://mlsanity.com/paper/2409.06916/) (2024)
- [Understanding Distribution Structure on Calibrated Recommendation Systems](https://mlsanity.com/paper/2508.13568/) (2025)

## Abstract (arXiv)

Despite the benefits of personalizing items and information tailored to users' needs, it has been found that recommender systems tend to introduce biases that favor popular items or certain categories of items, and dominant user groups. In this study, we aim to characterize the systematic errors of a recommendation system and how they manifest in various accountability issues, such as stereotypes, biases, and miscalibration. We propose a unified framework that distinguishes the sources of prediction errors into a set of key measures that quantify the various types of system-induced effects, both at the individual and collective levels. Based on our measuring framework, we examine the most widely adopted algorithms in the context of movie recommendation. Our research reveals three important findings: (1) Differences between algorithms: recommendations generated by simpler algorithms tend to be more stereotypical but less biased than those generated by more complex algorithms. (2) Disparate impact on groups and individuals: system-induced biases and stereotypes have a disproportionate effect on atypical users and minority groups (e.g., women and older users). (3) Mitigation opportunity: using structural equation modeling, we identify the interactions between user characteristics (typicality and diversity), system-induced effects, and miscalibration. We further investigate the possibility of mitigating system-induced effects by oversampling underrepresented groups and individuals, which was found to be effective in reducing stereotypes and improving recommendation quality. Our research is the first systematic examination of not only system-induced effects and miscalibration but also the stereotyping issue in recommender systems.