# Explanatory Debiasing: Involving Domain Experts in the Data Generation Process to Mitigate Representation Bias in AI Systems

arXiv: https://arxiv.org/abs/2501.01441 · HTML: https://mlsanity.com/paper/2501.01441/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 74 references that are in the map.
- First posted on arXiv 26 Dec 2024.

## Builds on (most cited first, 24 of 24)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Metrics for Explainable AI: Challenges and Prospects](https://mlsanity.com/paper/1812.04608/) (2018)
- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [Making deep neural networks right for the right scientific reasons by interacting with their explanations](https://mlsanity.com/paper/2001.05371/) (2020)
- [Learning to Compose Domain-Specific Transformations for Data Augmentation](https://mlsanity.com/paper/1709.01643/) (2017)
- [Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS](https://mlsanity.com/paper/1912.06059/) (2019)
- [The Effects of Regularization and Data Augmentation are Class Dependent](https://mlsanity.com/paper/2204.03632/) (2022)
- [TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations](https://mlsanity.com/paper/2207.04154/) (2022)
- [Leakage and the Reproducibility Crisis in ML-based Science](https://mlsanity.com/paper/2207.07048/) (2022)
- [Incorporating Domain Knowledge into Deep Neural Networks](https://mlsanity.com/paper/2103.00180/) (2021)
- [Representation Bias in Data: A Survey on Identification and Resolution Techniques](https://mlsanity.com/paper/2203.11852/) (2022)
- [OnlineAugment: Online Data Augmentation with Less Domain Knowledge](https://mlsanity.com/paper/2007.09271/) (2020)
- [Directive Explanations for Monitoring the Risk of Diabetes Onset: Introducing Directive Data-Centric Explanations and Combinations to Support What-If Explorations](https://mlsanity.com/paper/2302.10671/) (2023)
- [To SMOTE, or not to SMOTE?](https://mlsanity.com/paper/2201.08528/) (2022)
- [EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations](https://mlsanity.com/paper/2402.00491/) (2024)
- [Solving the Class Imbalance Problem Using a Counterfactual Method for Data Augmentation](https://mlsanity.com/paper/2111.03516/) (2021)
- [Towards Directive Explanations: Crafting Explainable AI Systems for Actionable Human-AI Interactions](https://mlsanity.com/paper/2401.04118/) (2023)
- [Lessons Learned from EXMOS User Studies: A Technical Report Summarizing Key Takeaways from User Studies Conducted to Evaluate The EXMOS Platform](https://mlsanity.com/paper/2310.02063/) (2023)
- [An Explanatory Model Steering System for Collaboration between Domain Experts and AI](https://mlsanity.com/paper/2405.13038/) (2024)
- [Representation Debiasing of Generated Data Involving Domain Experts](https://mlsanity.com/paper/2407.09485/) (2024)
- [Data Coverage for Detecting Representation Bias in Image Datasets: A Crowdsourcing Approach](https://mlsanity.com/paper/2306.13868/) (2023)
- [Data augmentation and explainability for bias discovery and mitigation in deep learning](https://mlsanity.com/paper/2308.09464/) (2023)

## Built on by (most cited first, 5 of 5)

- [Show Me How: Benefits and Challenges of Agent-Augmented Counterfactual Explanations for Non-Expert Users](https://mlsanity.com/paper/2504.13897/) (2025)
- [Let's Get You Hired: A Job Seeker's Perspective on Multi-Agent Recruitment Systems for Explaining Hiring Decisions](https://mlsanity.com/paper/2505.20312/) (2025)
- [Emergent, not Immanent: A Baradian Reading of Explainable AI](https://mlsanity.com/paper/2601.15029/) (2026)
- [What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Research](https://mlsanity.com/paper/2608.10431/) (2026)
- [ContextBias: Controlled Evaluation of Bias Persistence Under Context Shift in Text-to-Image Models](https://mlsanity.com/paper/2608.29847/) (2026)

## Abstract (arXiv)

Representation bias is one of the most common types of biases in artificial intelligence (AI) systems, causing AI models to perform poorly on underrepresented data segments. Although AI practitioners use various methods to reduce representation bias, their effectiveness is often constrained by insufficient domain knowledge in the debiasing process. To address this gap, this paper introduces a set of generic design guidelines for effectively involving domain experts in representation debiasing. We instantiated our proposed guidelines in a healthcare-focused application and evaluated them through a comprehensive mixed-methods user study with 35 healthcare experts. Our findings show that involving domain experts can reduce representation bias without compromising model accuracy. Based on our findings, we also offer recommendations for developers to build robust debiasing systems guided by our generic design guidelines, ensuring more effective inclusion of domain experts in the debiasing process.