# False perfection in machine prediction: Detecting and assessing circularity problems in machine learning

arXiv: https://arxiv.org/abs/2106.12417 · HTML: https://mlsanity.com/paper/2106.12417/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 44 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 11 of 11)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Neural Additive Models: Interpretable Machine Learning with Neural Nets](https://mlsanity.com/paper/2004.13912/) (2020)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Learning Not to Learn: Training Deep Neural Networks with Biased Data](https://mlsanity.com/paper/1812.10352/) (2018)
- [Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation](https://mlsanity.com/paper/1710.06169/) (2017)
- [Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models](https://mlsanity.com/paper/2005.14709/) (2020)
- [Leveraging Implicit Expert Knowledge for Non-Circular Machine Learning in Sepsis Prediction](https://mlsanity.com/paper/1909.09557/) (2019)
- [Embedding Meta-Textual Information for Improved Learning to Rank](https://mlsanity.com/paper/2010.16313/) (2020)

## Abstract (arXiv)

This paper is an excerpt of an early version of Chapter 2 of the book "Validity, Reliability, and Significance. Empirical Methods for NLP and Data Science", by Stefan Riezler and Michael Hagmann, published in December 2021 by Morgan & Claypool. Please see the book's homepage at https://www.morganclaypoolpublishers.com/catalog_Orig/product_info.php?products_id=1688 for a more recent and comprehensive discussion.