mlsanityData to 7 Oct 2026

Fairness Testing: Testing Software for Discrimination

arXiv 1709.03221 ↗First posted 11 Sep 2017cs.SE cs.AI cs.CYcited by 123

Not on the map. Why?

Abstract (arXiv)

This paper defines software fairness and discrimination and develops a testing-based method for measuring if and how much software discriminates, focusing on causality in discriminatory behavior. Evidence of software discrimination has been found in modern software systems that recommend criminal sentences, grant access to financial products, and determine who is allowed to participate in promotions. Our approach, Themis, generates efficient test suites to measure discrimination. Given a schema describing valid system inputs, Themis generates discrimination tests automatically and does not require an oracle. We evaluate Themis on 20 software systems, 12 of which come from prior work with explicit focus on avoiding discrimination. We find that (1) Themis is effective at discovering software discrimination, (2) state-of-the-art techniques for removing discrimination from algorithms fail in many situations, at times discriminating against as much as 98% of an input subdomain, (3) Themis optimizations are effective at producing efficient test suites for measuring discrimination, and (4) Themis is more efficient on systems that exhibit more discrimination. We thus demonstrate that fairness testing is a critical aspect of the software development cycle in domains with possible discrimination and provide initial tools for measuring software discrimination.

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 27; 2022: 25; 2023: 18; 2024: 25; 2025: 19; 2026: 9.

123 papers.

Sort:

By area:

  1. 2021Trustworthy AI: From Principles to Practicescited by 206
  2. 2023Baldur: Whole-Proof Generation and Repair with Large Language Modelscited by 118
  3. 2021Bias in Machine Learning Software: Why? How? What to do?cited by 92
  4. 2021Socially Responsible AI Algorithms: Issues, Purposes, and Challengescited by 56
  5. 2021Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactualscited by 55
  6. 2021Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipelinecited by 49
  7. 2021NeuronFair: Interpretable White-Box Fairness Testing through Biased Neuron Identificationcited by 42
  8. 2022Representation Bias in Data: A Survey on Identification and Resolution Techniquescited by 37
  9. 2021A Review of Formal Methods applied to Machine Learningcited by 33
  10. 2024GenderBias-VL: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probingcited by 31
  11. 2022A Comprehensive Empirical Study of Bias Mitigation Methods for Machine Learning Classifierscited by 30
  12. 2022Fairness-aware Configuration of Machine Learning Librariescited by 28
38 more
  1. 2022Fairness in Recommendation: Foundations, Methods and Applicationscited by 27
  2. 2024Rango: Adaptive Retrieval-Augmented Proving for Automated Software Verificationcited by 25
  3. 2023Latent Imitator: Generating Natural Individual Discriminatory Instances for Black-Box Fairness Testingcited by 23
  4. 2022Fairify: Fairness Verification of Neural Networkscited by 22
  5. 2021Technical Challenges for Training Fair Neural Networkscited by 20
  6. 2023Bias and Fairness in Chatbots: An Overviewcited by 20
  7. 2021FairMask: Better Fairness via Model-based Rebalancing of Protected Attributescited by 19
  8. 2024Few-Shot Fairness: Unveiling LLM's Potential for Fairness-Aware Classificationcited by 18
  9. 2021Removing biased data to improve fairness and accuracycited by 17
  10. 2021Probabilistic Verification of Neural Networks Against Group Fairnesscited by 17
  11. 2022Adaptive Fairness Improvement Based on Causality Analysiscited by 17
  12. 2022Towards Understanding Fairness and its Composition in Ensemble Machine Learningcited by 17
  13. 2022Accurate Fairness: Improving Individual Fairness without Trading Accuracycited by 16
  14. 2021Automatic Fairness Testing of Neural Classifiers through Adversarial Samplingcited by 14
  15. 2021Fair Enough: Searching for Sufficient Measures of Fairnesscited by 14
  16. 2021Through the Data Management Lens: Experimental Analysis and Evaluation of Fair Classificationcited by 13
  17. 2024The Pursuit of Fairness in Artificial Intelligence Models: A Surveycited by 13
  18. 2022Towards Involving End-users in Interactive Human-in-the-loop AI Fairnesscited by 12
  19. 2021MDPFuzz: Testing Models Solving Markov Decision Processescited by 11
  20. 2023Information-Theoretic Testing and Debugging of Fairness Defects in Deep Neural Networkscited by 10
  21. 2024Fairness Definitions in Language Models Explainedcited by 10
  22. 2021AequeVox: Automated Fairness Testing of Speech Recognition Systemscited by 9
  23. 2023Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoMLcited by 9
  24. 2024NeuFair: Neural Network Fairness Repair with Dropoutcited by 9
  25. 2023Counterfactual Situation Testing: Uncovering Discrimination under Fairness given the Differencecited by 8
  26. 2024Cobblestone: A Divide-and-Conquer Approach for Automating Formal Verificationcited by 8
  27. 2022TESTSGD: Interpretable Testing of Neural Networks Against Subtle Group Discriminationcited by 7
  28. 2021Fairness Testing of Deep Image Classification with Adequacy Metricscited by 6
  29. 2022Provable Fairness for Neural Network Models using Formal Verificationcited by 6
  30. 2024A Large-Scale Empirical Study on Improving the Fairness of Image Classification Modelscited by 6
  31. 2022Specifying and Testing k-Safety Properties for Machine-Learning Modelscited by 5
  32. 2022Explainable Global Fairness Verification of Tree-Based Classifierscited by 5
  33. 2023Adversarial Sampling for Fairness Testing in Deep Neural Networkcited by 5
  34. 2024What is Fair? Defining Fairness in Machine Learning for Healthcited by 5
  35. 2021Learning to Generate Fair Clusters from Demonstrationscited by 4
  36. 2021Quality Assurance Challenges for Machine Learning Software Applications During Software Development Life Cycle Phasescited by 4
  37. 2021Fair-SSL: Building fair ML Software with less datacited by 4
  38. 2022PrivFair: a Library for Privacy-Preserving Fairness Auditingcited by 4

50 most cited of 123 All 123 (JSON)