# FaceSaliencyAug: Mitigating Geographic, Gender and Stereotypical Biases via Saliency-Based Data Augmentation

arXiv: https://arxiv.org/abs/2410.14070 · HTML: https://mlsanity.com/paper/2410.14070/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 52 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 25 of 25)

- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-supervised Object and Action Localization](https://mlsanity.com/paper/1704.04232/) (2017)
- [SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularization](https://mlsanity.com/paper/2006.01791/) (2020)
- [GridMask Data Augmentation](https://mlsanity.com/paper/2001.04086/) (2020)
- [Learning Debiased Representation via Disentangled Feature Augmentation](https://mlsanity.com/paper/2107.01372/) (2021)
- [BiaSwap: Removing dataset bias with bias-tailored swapping augmentation](https://mlsanity.com/paper/2108.10008/) (2021)
- [Towards Accuracy-Fairness Paradox: Adversarial Example-based Data Augmentation for Visual Debiasing](https://mlsanity.com/paper/2007.13632/) (2020)
- [Image Data Augmentation Approaches: A Comprehensive Survey and Future directions](https://mlsanity.com/paper/2301.02830/) (2023)
- [Multimodal Composite Association Score: Measuring Gender Bias in Generative Multimodal Models](https://mlsanity.com/paper/2304.13855/) (2023)
- [Implicit Diversity in Image Summarization](https://mlsanity.com/paper/1901.10265/) (2019)
- [Investigating Multi-Feature Selection and Ensembling for Audio Classification](https://mlsanity.com/paper/2206.07511/) (2022)
- [Precise Single-stage Detector](https://mlsanity.com/paper/2210.04252/) (2022)
- [Random Data Augmentation based Enhancement: A Generalized Enhancement Approach for Medical Datasets](https://mlsanity.com/paper/2210.00824/) (2022)
- [Understanding EEG signals for subject-wise Definition of Armoni Activities](https://mlsanity.com/paper/2301.00948/) (2023)
- [Gender Bias in Multimodal Models: A Transnational Feminist Approach Considering Geographical Region and Culture](https://mlsanity.com/paper/2309.04997/) (2023)
- [Biased Attention: Do Vision Transformers Amplify Gender Bias More than Convolutional Neural Networks?](https://mlsanity.com/paper/2309.08760/) (2023)
- [FocusedDropout for Convolutional Neural Network](https://mlsanity.com/paper/2103.15425/) (2021)
- [AudRandAug: Random Image Augmentations for Audio Classification](https://mlsanity.com/paper/2309.04762/) (2023)
- [KeepOriginalAugment: Single Image-based Better Information-Preserving Data Augmentation Approach](https://mlsanity.com/paper/2405.06354/) (2024)
- [OxML Challenge 2023: Carcinoma classification using data augmentation](https://mlsanity.com/paper/2409.10544/) (2024)

## Abstract (arXiv)

Geographical, gender and stereotypical biases in computer vision models pose significant challenges to their performance and fairness. {In this study, we present an approach named FaceSaliencyAug aimed at addressing the gender bias in} {Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). Leveraging the salient regions} { of faces detected by saliency, the propose approach mitigates geographical and stereotypical biases } {in the datasets. FaceSaliencyAug} randomly selects masks from a predefined search space and applies them to the salient region of face images, subsequently restoring the original image with masked salient region. {The proposed} augmentation strategy enhances data diversity, thereby improving model performance and debiasing effects. We quantify dataset diversity using Image Similarity Score (ISS) across five datasets, including Flickr Faces HQ (FFHQ), WIKI, IMDB, Labelled Faces in the Wild (LFW), UTK Faces, and Diverse Dataset. The proposed approach demonstrates superior diversity metrics, as evaluated by ISS-intra and ISS-inter algorithms. Furthermore, we evaluate the effectiveness of our approach in mitigating gender bias on CEO, Engineer, Nurse, and School Teacher datasets. We use the Image-Image Association Score (IIAS) to measure gender bias in these occupations. Our experiments reveal a reduction in gender bias for both CNNs and ViTs, indicating the efficacy of our method in promoting fairness and inclusivity in computer vision models.