# DASH: Visual Analytics for Debiasing Image Classification via User-Driven Synthetic Data Augmentation

arXiv: https://arxiv.org/abs/2209.06357 · HTML: https://mlsanity.com/paper/2209.06357/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 14 Sep 2022.

## Builds on (most cited first, 15 of 15)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [Learning De-biased Representations with Biased Representations](https://mlsanity.com/paper/1910.02806/) (2019)
- [Grad-CAM: Why did you say that?](https://mlsanity.com/paper/1611.07450/) (2016)
- [STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing](https://mlsanity.com/paper/1904.09709/) (2019)
- [Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias](https://mlsanity.com/paper/2001.03152/) (2020)
- [Embedding Projector: Interactive Visualization and Interpretation of Embeddings](https://mlsanity.com/paper/1611.05469/) (2016)
- [XGAN: Unsupervised Image-to-Image Translation for Many-to-Many Mappings](https://mlsanity.com/paper/1711.05139/) (2017)
- [t-SNE-CUDA: GPU-Accelerated t-SNE and its Applications to Modern Data](https://mlsanity.com/paper/1807.11824/) (2018)
- [Exploring Unlabeled Faces for Novel Attribute Discovery](https://mlsanity.com/paper/1912.03085/) (2019)

## Built on by (most cited first, 7 of 7)

- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [Finspector: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models](https://mlsanity.com/paper/2305.16937/) (2023)
- [ASAP: Interpretable Analysis and Summarization of AI-generated Image Patterns at Scale](https://mlsanity.com/paper/2404.02990/) (2024)
- [Enhancing Intrinsic Features for Debiasing via Investigating Class-Discerning Common Attributes in Bias-Contrastive Pair](https://mlsanity.com/paper/2404.19250/) (2024)
- [ESCAPE: Countering Systematic Errors from Machine's Blind Spots via Interactive Visual Analysis](https://mlsanity.com/paper/2303.09657/) (2023)
- [MiMICRI: Towards Domain-centered Counterfactual Explanations of Cardiovascular Image Classification Models](https://mlsanity.com/paper/2404.16174/) (2024)
- [Interactive Counterfactual Exploration of Algorithmic Harms in Recommender Systems](https://mlsanity.com/paper/2409.06916/) (2024)

## Abstract (arXiv)

Image classification models often learn to predict a class based on irrelevant co-occurrences between input features and an output class in training data. We call the unwanted correlations "data biases," and the visual features causing data biases "bias factors." It is challenging to identify and mitigate biases automatically without human intervention. Therefore, we conducted a design study to find a human-in-the-loop solution. First, we identified user tasks that capture the bias mitigation process for image classification models with three experts. Then, to support the tasks, we developed a visual analytics system called DASH that allows users to visually identify bias factors, to iteratively generate synthetic images using a state-of-the-art image-to-image translation model, and to supervise the model training process for improving the classification accuracy. Our quantitative evaluation and qualitative study with ten participants demonstrate the usefulness of DASH and provide lessons for future work.