# Algorithms are not neutral: Bias in collaborative filtering

arXiv: https://arxiv.org/abs/2105.01031 · HTML: https://mlsanity.com/paper/2105.01031/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 55 references that are in the map.
- First posted on arXiv 3 May 2021.

## Builds on (most cited first, 5 of 5)

- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [The Unfairness of Popularity Bias in Recommendation](https://mlsanity.com/paper/1907.13286/) (2019)
- [What does it mean to solve the problem of discrimination in hiring? Social, technical and legal perspectives from the UK on automated hiring systems](https://mlsanity.com/paper/1910.06144/) (2019)
- [Exploring Author Gender in Book Rating and Recommendation](https://mlsanity.com/paper/1808.07586/) (2018)
- [Auditing Search Engines for Differential Satisfaction Across Demographics](https://mlsanity.com/paper/1705.10689/) (2017)

## Built on by (most cited first, 3 of 3)

- [Meaningful human control: actionable properties for AI system development](https://mlsanity.com/paper/2112.01298/) (2021)
- [Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness](https://mlsanity.com/paper/2110.08353/) (2021)
- [Can Artificial Intelligence Embody Moral Values?](https://mlsanity.com/paper/2408.12250/) (2024)

## Abstract (arXiv)

Discussions of algorithmic bias tend to focus on examples where either the data or the people building the algorithms are biased. This gives the impression that clean data and good intentions could eliminate bias. The neutrality of the algorithms themselves is defended by prominent Artificial Intelligence researchers. However, algorithms are not neutral. In addition to biased data and biased algorithm makers, AI algorithms themselves can be biased. This is illustrated with the example of collaborative filtering, which is known to suffer from popularity, and homogenizing biases. Iterative information filtering algorithms in general create a selection bias in the course of learning from user responses to documents that the algorithm recommended. These are not merely biases in the statistical sense; these statistical biases can cause discriminatory outcomes. Data points on the margins of distributions of human data tend to correspond to marginalized people. Popularity and homogenizing biases have the effect of further marginalizing the already marginal. This source of bias warrants serious attention given the ubiquity of algorithmic decision-making.