# Emergent Instabilities in Algorithmic Feedback Loops

arXiv: https://arxiv.org/abs/2201.07203 · HTML: https://mlsanity.com/paper/2201.07203/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 18 Jan 2022.

## Builds on (most cited first, 18 of 18)

- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Implicit Regularization in Matrix Factorization](https://mlsanity.com/paper/1705.09280/) (2017)
- [Reinforcement learning based recommender systems: A survey](https://mlsanity.com/paper/2101.06286/) (2021)
- [Fairness in Learning: Classic and Contextual Bandits](https://mlsanity.com/paper/1605.07139/) (2016)
- [How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility](https://mlsanity.com/paper/1710.11214/) (2017)
- [Runaway Feedback Loops in Predictive Policing](https://mlsanity.com/paper/1706.09847/) (2017)
- [Fairness in Reinforcement Learning](https://mlsanity.com/paper/1611.03071/) (2016)
- [Feedback Loop and Bias Amplification in Recommender Systems](https://mlsanity.com/paper/2007.13019/) (2020)
- [An analytic theory of generalization dynamics and transfer learning in deep linear networks](https://mlsanity.com/paper/1809.10374/) (2018)
- [Understanding Echo Chambers in E-commerce Recommender Systems](https://mlsanity.com/paper/2007.02474/) (2020)
- [The Use of Machine Learning Algorithms in Recommender Systems: A Systematic Review](https://mlsanity.com/paper/1511.05263/) (2015)
- [Deconvolving Feedback Loops in Recommender Systems](https://mlsanity.com/paper/1703.01049/) (2017)
- [Partially Observable Markov Decision Process for Recommender Systems](https://mlsanity.com/paper/1608.07793/) (2016)
- [Users Polarization on Facebook and Youtube](https://mlsanity.com/paper/1604.02705/) (2016)
- [Algorithmic bias amplifies opinion polarization: A bounded confidence model](https://mlsanity.com/paper/1803.02111/) (2018)
- [Origins of Algorithmic Instabilities in Crowdsourced Ranking](https://mlsanity.com/paper/2010.12571/) (2020)
- [GuessTheKarma: A Game to Assess Social Rating Systems](https://mlsanity.com/paper/1809.00740/) (2018)

## Built on by (most cited first, 1 of 1)

- [Diversified Recommendations for Agents with Adaptive Preferences](https://mlsanity.com/paper/2210.07773/) (2022)

## Abstract (arXiv)

Algorithms that aid human tasks, such as recommendation systems, are ubiquitous. They appear in everything from social media to streaming videos to online shopping. However, the feedback loop between people and algorithms is poorly understood and can amplify cognitive and social biases (algorithmic confounding), leading to unexpected outcomes. In this work, we explore algorithmic confounding in collaborative filtering-based recommendation algorithms through teacher-student learning simulations. Namely, a student collaborative filtering-based model, trained on simulated choices, is used by the recommendation algorithm to recommend items to agents. Agents might choose some of these items, according to an underlying teacher model, with new choices then fed back into the student model as new training data (approximating online machine learning). These simulations demonstrate how algorithmic confounding produces erroneous recommendations which in turn lead to instability, i.e., wide variations in an item's popularity between each simulation realization. We use the simulations to demonstrate a novel approach to training collaborative filtering models that can create more stable and accurate recommendations. Our methodology is general enough that it can be extended to other socio-technical systems in order to better quantify and improve the stability of algorithms. These results highlight the need to account for emergent behaviors from interactions between people and algorithms.