# Analysis of hidden feedback loops in continuous machine learning systems

arXiv: https://arxiv.org/abs/2101.05673 · HTML: https://mlsanity.com/paper/2101.05673/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 19 references that are in the map.
- First posted on arXiv 14 Jan 2021.

## Builds on (most cited first, 3 of 3)

- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [Runaway Feedback Loops in Predictive Policing](https://mlsanity.com/paper/1706.09847/) (2017)

## Built on by (most cited first, 2 of 2)

- [MLDev: Data Science Experiment Automation and Reproducibility Software](https://mlsanity.com/paper/2107.12322/) (2021)
- [Existence conditions for hidden feedback loops in online recommender systems](https://mlsanity.com/paper/2109.05278/) (2021)

## Abstract (arXiv)

In this concept paper, we discuss intricacies of specifying and verifying the quality of continuous and lifelong learning artificial intelligence systems as they interact with and influence their environment causing a so-called concept drift. We signify a problem of implicit feedback loops, demonstrate how they intervene with user behavior on an exemplary housing prices prediction system. Based on a preliminary model, we highlight conditions when such feedback loops arise and discuss possible solution approaches.