mlsanityData to 14 Oct 2026

Distribution-aware Fairness Test Generation

arXiv 2305.13935 ↗First posted 8 May 2023cs.CV cs.LG cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

Ensuring that all classes of objects are detected with equal accuracy is essential in AI systems. For instance, being unable to identify any one class of objects could have fatal consequences in autonomous driving systems. Hence, ensuring the reliability of image recognition systems is crucial. This work addresses how to validate group fairness in image recognition software. We propose a distribution-aware fairness testing approach (called DistroFair) that systematically exposes class-level fairness violations in image classifiers via a synergistic combination of out-of-distribution (OOD) testing and semantic-preserving image mutation. DistroFair automatically learns the distribution (e.g., number/orientation) of objects in a set of images. Then it systematically mutates objects in the images to become OOD using three semantic-preserving image mutations - object deletion, object insertion and object rotation. We evaluate DistroFair using two well-known datasets (CityScapes and MS-COCO) and three major, commercial image recognition software (namely, Amazon Rekognition, Google Cloud Vision and Azure Computer Vision). Results show that about 21% of images generated by DistroFair reveal class-level fairness violations using either ground truth or metamorphic oracles. DistroFair is up to 2.3x more effective than two main baselines, i.e., (a) an approach which focuses on generating images only within the distribution (ID) and (b) fairness analysis using only the original image dataset. We further observed that DistroFair is efficient, it generates 460 images per hour, on average. Finally, we evaluate the semantic validity of our approach via a user study with 81 participants, using 30 real images and 30 corresponding mutated images generated by DistroFair. We found that images generated by DistroFair are 80% as realistic as real-world images.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 77 references that are in the map. First posted on arXiv 8 May 2023.

Builds on

39 of 77 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  4. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  5. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
  6. 2018Panoptic Segmentationcited by 802
  7. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  8. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  9. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 339
  10. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 336
  11. 2017DeepXplore: Automated Whitebox Testing of Deep Learning Systemscited by 311
  12. 2018Learning Not to Learn: Training Deep Neural Networks with Biased Datacited by 255
27 more
  1. 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
  2. 2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 251
  3. 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
  4. 2019Does Object Recognition Work for Everyone?cited by 151
  5. 2022Bias Mitigation for Machine Learning Classifiers: A Comprehensive Surveycited by 125
  6. 2018ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositingcited by 90
  7. 2021OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalizationcited by 86
  8. 2018Automated Directed Fairness Testingcited by 78
  9. 2020On the Value of Out-of-Distribution Testing: An Example of Goodhart's Lawcited by 74
  10. 2022Fairness Testing: A Comprehensive Survey and Analysis of Trendscited by 62
  11. 2019Out-of-distribution Detection in Classifiers via Generationcited by 49
  12. 2019Automatic Testing and Improvement of Machine Translationcited by 36
  13. 2022PyTorch Image Quality: Metrics for Image Quality Assessmentcited by 34
  14. 2022Fairness-aware Configuration of Machine Learning Librariescited by 28
  15. 2022Improving the Intra-class Long-tail in 3D Detection via Rare Example Miningcited by 19
  16. 2019Testing DNN Image Classifiers for Confusion & Bias Errorscited by 18
  17. 2019Age and gender bias in pedestrian detection algorithmscited by 18
  18. 2021Automatic Fairness Testing of Neural Classifiers through Adversarial Samplingcited by 14
  19. 2021BiasRV: Uncovering Biased Sentiment Predictions at Runtimecited by 12
  20. 2022Automated Testing of Image Captioning Systemscited by 12
  21. 2020Astraea: Grammar-based Fairness Testingcited by 11
  22. 2021FreaAI: Automated extraction of data slices to test machine learning modelscited by 10
  23. 2022Hierarchical Distribution-Aware Testing of Deep Learningcited by 8
  24. 2022Software Fairness: An Analysis and Surveycited by 8
  25. 2021Distribution Awareness for AI System Testingcited by 6
  26. 2021Fairness Testing of Deep Image Classification with Adequacy Metricscited by 6
  27. 2021A framework for the automation of testing computer vision systemscited by 1

Built on by

Not cited yet.