# Representation Bias in Data: A Survey on Identification and Resolution Techniques

arXiv: https://arxiv.org/abs/2203.11852 · HTML: https://mlsanity.com/paper/2203.11852/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 74 of its 183 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 30 of 74)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Mitigating Gender Bias in Natural Language Processing: Literature Review](https://mlsanity.com/paper/1906.08976/) (2019)
- [A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle](https://mlsanity.com/paper/1901.10002/) (2019)
- [Censoring Representations with an Adversary](https://mlsanity.com/paper/1511.05897/) (2015)
- [Learning Gender-Neutral Word Embeddings](https://mlsanity.com/paper/1809.01496/) (2018)
- [Fairness in Learning: Classic and Contextual Bandits](https://mlsanity.com/paper/1605.07139/) (2016)
- [Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations](https://mlsanity.com/paper/1707.00075/) (2017)
- [FairGAN: Fairness-aware Generative Adversarial Networks](https://mlsanity.com/paper/1805.11202/) (2018)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [Reducing Gender Bias in Abusive Language Detection](https://mlsanity.com/paper/1808.07231/) (2018)
- [Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings](https://mlsanity.com/paper/1904.04047/) (2019)
- [Identifying and Correcting Label Bias in Machine Learning](https://mlsanity.com/paper/1901.04966/) (2019)
- [Why Is My Classifier Discriminatory?](https://mlsanity.com/paper/1805.12002/) (2018)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)

## Built on by (most cited first, 30 of 37)

- [Black-Box Access is Insufficient for Rigorous AI Audits](https://mlsanity.com/paper/2401.14446/) (2024)
- [Sociodemographic Bias in Language Models: A Survey and Forward Path](https://mlsanity.com/paper/2306.08158/) (2023)
- [Data Readiness for AI: A 360-Degree Survey](https://mlsanity.com/paper/2404.05779/) (2024)
- [FairAIED: Navigating Fairness, Bias, and Ethics in Educational AI Applications](https://mlsanity.com/paper/2407.18745/) (2024)
- [Visual-TCAV: Concept-based Attribution and Saliency Maps for Post-hoc Explainability in Image Classification](https://mlsanity.com/paper/2411.05698/) (2024)
- [Through the Fairness Lens: Experimental Analysis and Evaluation of Entity Matching](https://mlsanity.com/paper/2307.02726/) (2023)
- [Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey](https://mlsanity.com/paper/2310.17903/) (2023)
- [Consistent Range Approximation for Fair Predictive Modeling](https://mlsanity.com/paper/2212.10839/) (2022)
- [Chameleon: Foundation Models for Fairness-aware Multi-modal Data Augmentation to Enhance Coverage of Minorities](https://mlsanity.com/paper/2402.01071/) (2024)
- [Explanatory Debiasing: Involving Domain Experts in the Data Generation Process to Mitigate Representation Bias in AI Systems](https://mlsanity.com/paper/2501.01441/) (2024)
- [A Survey on Image-text Multimodal Models](https://mlsanity.com/paper/2309.15857/) (2023)
- [Survey on AI Ethics: A Socio-technical Perspective](https://mlsanity.com/paper/2311.17228/) (2023)
- [BIASINSPECTOR: Detecting Bias in Structured Data through LLM Agents](https://mlsanity.com/paper/2504.04855/) (2025)
- [Ethical Concern Identification in NLP: A Corpus of ACL Anthology Ethics Statements](https://mlsanity.com/paper/2411.07845/) (2024)
- [Best Practices for Responsible Machine Learning in Credit Scoring](https://mlsanity.com/paper/2409.20536/) (2024)
- [ColFigPhotoAttnNet: Reliable Finger Photo Presentation Attack Detection Leveraging Window-Attention on Color Spaces](https://mlsanity.com/paper/2503.05247/) (2025)
- [A Unified Framework to Quantify Cultural Intelligence of AI](https://mlsanity.com/paper/2603.01211/) (2026)
- [FairEM360: A Suite for Responsible Entity Matching](https://mlsanity.com/paper/2404.07354/) (2024)
- [Overcoming Representation Bias in Fairness-Aware data Repair using Optimal Transport](https://mlsanity.com/paper/2410.02840/) (2024)
- [Mining the Minoria: Unknown, Under-represented, and Under-performing Minority Groups](https://mlsanity.com/paper/2411.04761/) (2024)
- [Prompting Away Stereotypes? Evaluating Bias in Text-to-Image Models for Occupations](https://mlsanity.com/paper/2509.00849/) (2025)
- [Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanations](https://mlsanity.com/paper/2509.04515/) (2025)
- [Unbiased Binning for Fairness-aware Attribute Representation](https://mlsanity.com/paper/2509.21785/) (2025)
- [Justice in Healthcare Artificial Intelligence in Africa](https://mlsanity.com/paper/2406.10653/) (2024)
- [Dataset Representativeness and Downstream Task Fairness](https://mlsanity.com/paper/2407.00170/) (2024)
- [Leveraging Ontologies to Document Bias in Data](https://mlsanity.com/paper/2407.00509/) (2024)
- [Adaptive Recruitment Resource Allocation to Improve Cohort Representativeness in Participatory Biomedical Datasets](https://mlsanity.com/paper/2408.01375/) (2024)
- [Responsible AI in NLP: GUS-Net Span-Level Bias Detection Dataset and Benchmark for Generalizations, Unfairness, and Stereotypes](https://mlsanity.com/paper/2410.08388/) (2024)
- [Data Acquisition for Improving Model Fairness using Reinforcement Learning](https://mlsanity.com/paper/2412.03009/) (2024)
- [TEDI: Trustworthy and Ethical Dataset Indicators to Analyze and Compare Dataset Documentation](https://mlsanity.com/paper/2505.17841/) (2025)

## Abstract (arXiv)

Data-driven algorithms are only as good as the data they work with, while data sets, especially social data, often fail to represent minorities adequately. Representation Bias in data can happen due to various reasons ranging from historical discrimination to selection and sampling biases in the data acquisition and preparation methods. Given that "bias in, bias out", one cannot expect AI-based solutions to have equitable outcomes for societal applications, without addressing issues such as representation bias. While there has been extensive study of fairness in machine learning models, including several review papers, bias in the data has been less studied. This paper reviews the literature on identifying and resolving representation bias as a feature of a data set, independent of how consumed later. The scope of this survey is bounded to structured (tabular) and unstructured (e.g., image, text, graph) data. It presents taxonomies to categorize the studied techniques based on multiple design dimensions and provides a side-by-side comparison of their properties. There is still a long way to fully address representation bias issues in data. The authors hope that this survey motivates researchers to approach these challenges in the future by observing existing work within their respective domains.