# Fairness-aware Configuration of Machine Learning Libraries arXiv: https://arxiv.org/abs/2202.06196 ยท HTML: https://mlsanity.com/paper/2202.06196/ ## Key facts - Cited by 28 papers in the map (data through 14 Oct 2026). - Builds on 10 of its 48 references that are in the map. - First posted on arXiv 13 Feb 2022. ## Builds on (most cited first, 10 of 10) - [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016) - [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018) - [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018) - [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019) - [Fairness Testing: Testing Software for Discrimination](https://mlsanity.com/paper/1709.03221/) (2017) - [Automated Directed Fairness Testing](https://mlsanity.com/paper/1807.00468/) (2018) - [Finding Faster Configurations using FLASH](https://mlsanity.com/paper/1801.02175/) (2018) - [Software Engineering for Fairness: A Case Study with Hyperparameter Optimization](https://mlsanity.com/paper/1905.05786/) (2019) - [Detecting and Understanding Real-World Differential Performance Bugs in Machine Learning Libraries](https://mlsanity.com/paper/2006.01991/) (2020) - [Differential Performance Debugging with Discriminant Regression Trees](https://mlsanity.com/paper/1711.04076/) (2017) ## Built on by (most cited first, 28 of 28) - [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022) - [Bias Testing and Mitigation in LLM-based Code Generation](https://mlsanity.com/paper/2309.14345/) (2023) - [Towards Understanding Fairness and its Composition in Ensemble Machine Learning](https://mlsanity.com/paper/2212.04593/) (2022) - [NeuFair: Neural Network Fairness Repair with Dropout](https://mlsanity.com/paper/2407.04268/) (2024) - [Fairness Testing of Large Language Models in Role-Playing](https://mlsanity.com/paper/2411.00585/) (2024) - [Information-Theoretic Testing and Debugging of Fairness Defects in Deep Neural Networks](https://mlsanity.com/paper/2304.04199/) (2023) - [Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoML](https://mlsanity.com/paper/2306.09297/) (2023) - [Understanding Fairness Surrogate Functions in Algorithmic Fairness](https://mlsanity.com/paper/2310.11211/) (2023) - [Causality-Aided Trade-off Analysis for Machine Learning Fairness](https://mlsanity.com/paper/2305.13057/) (2023) - [A Large-Scale Empirical Study on Improving the Fairness of Image Classification Models](https://mlsanity.com/paper/2401.03695/) (2024) - [FairLay-ML: Intuitive Debugging of Fairness in Data-Driven Social-Critical Software](https://mlsanity.com/paper/2407.01423/) (2024) - [FairSense: Long-Term Fairness Analysis of ML-Enabled Systems](https://mlsanity.com/paper/2501.01665/) (2025) - [FairLay-ML: Intuitive Remedies for Unfairness in Data-Driven Social-Critical Algorithms](https://mlsanity.com/paper/2307.05029/) (2023) - [Algorithmic Fairness: A Tolerance Perspective](https://mlsanity.com/paper/2405.09543/) (2024) - [Fairness Testing through Extreme Value Theory](https://mlsanity.com/paper/2501.11597/) (2025) - [Towards Better Fairness-Utility Trade-off: A Comprehensive Measurement-Based Reinforcement Learning Framework](https://mlsanity.com/paper/2307.11379/) (2023) - [Predicting Fairness of ML Software Configurations](https://mlsanity.com/paper/2404.19100/) (2024) - [Software Engineering Principles for Fairer Systems: Experiments with GroupCART](https://mlsanity.com/paper/2504.12587/) (2025) - [Fair-FLIP: Fair Deepfake Detection with Fairness-Oriented Final Layer Input Prioritising](https://mlsanity.com/paper/2507.08912/) (2025) - [Software Fairness Dilemma: Is Bias Mitigation a Zero-Sum Game?](https://mlsanity.com/paper/2508.03323/) (2025) - [Causally Perturbed Fairness Testing](https://mlsanity.com/paper/2510.18719/) (2025) - [Uncovering Discrimination Clusters: Quantifying and Explaining Systematic Fairness Violations](https://mlsanity.com/paper/2512.23769/) (2025) - [A Retrospective on ICSE 2022](https://mlsanity.com/paper/2207.12578/) (2022) - [Distribution-aware Fairness Test Generation](https://mlsanity.com/paper/2305.13935/) (2023) - [Contexts Matter: An Empirical Study on Contextual Influence in Fairness Testing for Deep Learning Systems](https://mlsanity.com/paper/2408.06102/) (2024) - [Whence Is A Model Fair? Fixing Fairness Bugs via Propensity Score Matching](https://mlsanity.com/paper/2504.17066/) (2025) - [A Generalised Exponentiated Gradient Approach to Enhance Fairness in Binary and Multi-class Classification Tasks](https://mlsanity.com/paper/2603.21393/) (2026) - [Fairness Invariants: A Relational Approach to Explaining and Mitigating Fairness Bugs](https://mlsanity.com/paper/2608.26209/) (2026) ## Abstract (arXiv) This paper investigates the parameter space of machine learning (ML) algorithms in aggravating or mitigating fairness bugs. Data-driven software is increasingly applied in social-critical applications where ensuring fairness is of paramount importance. The existing approaches focus on addressing fairness bugs by either modifying the input dataset or modifying the learning algorithms. On the other hand, the selection of hyperparameters, which provide finer controls of ML algorithms, may enable a less intrusive approach to influence the fairness. Can hyperparameters amplify or suppress discrimination present in the input dataset? How can we help programmers in detecting, understanding, and exploiting the role of hyperparameters to improve the fairness? We design three search-based software testing algorithms to uncover the precision-fairness frontier of the hyperparameter space. We complement these algorithms with statistical debugging to explain the role of these parameters in improving fairness. We implement the proposed approaches in the tool Parfait-ML (PARameter FAIrness Testing for ML Libraries) and show its effectiveness and utility over five mature ML algorithms as used in six social-critical applications. In these applications, our approach successfully identified hyperparameters that significantly improve (vis-a-vis the state-of-the-art techniques) the fairness without sacrificing precision. Surprisingly, for some algorithms (e.g., random forest), our approach showed that certain configuration of hyperparameters (e.g., restricting the search space of attributes) can amplify biases across applications. Upon further investigation, we found intuitive explanations of these phenomena, and the results corroborate similar observations from the literature.