# Detecting discriminatory risk through data annotation based on Bayesian inferences

arXiv: https://arxiv.org/abs/2101.11358 · HTML: https://mlsanity.com/paper/2101.11358/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 55 references that are in the map.
- First posted on arXiv 27 Jan 2021.

## Builds on (most cited first, 15 of 15)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning](https://mlsanity.com/paper/1912.10389/) (2019)
- [The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards](https://mlsanity.com/paper/1805.03677/) (2018)
- [Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing](https://mlsanity.com/paper/2001.00964/) (2020)
- [A Deeper Look at Dataset Bias](https://mlsanity.com/paper/1505.01257/) (2015)
- [Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From?](https://mlsanity.com/paper/1912.08320/) (2019)
- [Current Challenges and Visions in Music Recommender Systems Research](https://mlsanity.com/paper/1710.03208/) (2017)
- [Assessing and Remedying Coverage for a Given Dataset](https://mlsanity.com/paper/1810.06742/) (2018)
- [Identifying Bias in AI using Simulation](https://mlsanity.com/paper/1810.00471/) (2018)
- [The Geography of Pok'emon GO: Beneficial and Problematic Effects on Places and Movement](https://mlsanity.com/paper/1903.12041/) (2019)
- [Predicting class-imbalanced business risk using resampling, regularization, and model ensembling algorithms](https://mlsanity.com/paper/1903.05535/) (2019)
- [The invisible power of fairness. How machine learning shapes democracy](https://mlsanity.com/paper/1903.09493/) (2019)

## Built on by (most cited first, 7 of 7)

- [Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning](https://mlsanity.com/paper/2202.05338/) (2022)
- [Farsight: Fostering Responsible AI Awareness During AI Application Prototyping](https://mlsanity.com/paper/2402.15350/) (2024)
- [Ethical Considerations for Responsible Data Curation](https://mlsanity.com/paper/2302.03629/) (2023)
- [Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools](https://mlsanity.com/paper/2309.17337/) (2023)
- [Envisioning Narrative Intelligence: A Creative Visual Storytelling Anthology](https://mlsanity.com/paper/2310.04529/) (2023)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [The Perspective of Software Professionals on Algorithmic Racism](https://mlsanity.com/paper/2306.15133/) (2023)

## Abstract (arXiv)

Thanks to the increasing growth of computational power and data availability, the research in machine learning has advanced with tremendous rapidity. Nowadays, the majority of automatic decision making systems are based on data. However, it is well known that machine learning systems can present problematic results if they are built on partial or incomplete data. In fact, in recent years several studies have found a convergence of issues related to the ethics and transparency of these systems in the process of data collection and how they are recorded. Although the process of rigorous data collection and analysis is fundamental in the model design, this step is still largely overlooked by the machine learning community. For this reason, we propose a method of data annotation based on Bayesian statistical inference that aims to warn about the risk of discriminatory results of a given data set. In particular, our method aims to deepen knowledge and promote awareness about the sampling practices employed to create the training set, highlighting that the probability of success or failure conditioned to a minority membership is given by the structure of the data available. We empirically test our system on three datasets commonly accessed by the machine learning community and we investigate the risk of racial discrimination.