# An Empirical Study of Modular Bias Mitigators and Ensembles

arXiv: https://arxiv.org/abs/2202.00751 · HTML: https://mlsanity.com/paper/2202.00751/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 1 Feb 2022.

## Builds on (most cited first, 18 of 18)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees](https://mlsanity.com/paper/1806.06055/) (2018)
- [Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipeline](https://mlsanity.com/paper/2106.06054/) (2021)
- [Themis-ml: A Fairness-aware Machine Learning Interface for End-to-end Discrimination Discovery and Mitigation](https://mlsanity.com/paper/1710.06921/) (2017)
- [On Fairness, Diversity and Randomness in Algorithmic Decision Making](https://mlsanity.com/paper/1706.10208/) (2017)
- [FADE: FAir Double Ensemble Learning for Observable and Counterfactual Outcomes](https://mlsanity.com/paper/2109.00173/) (2021)
- [The Impact of Data Preparation on the Fairness of Software Systems](https://mlsanity.com/paper/1910.02321/) (2019)
- [An Empirical Study of Accuracy, Fairness, Explainability, Distributional Robustness, and Adversarial Robustness](https://mlsanity.com/paper/2109.14653/) (2021)

## Built on by (most cited first, 6 of 6)

- [Towards Understanding Fairness and its Composition in Ensemble Machine Learning](https://mlsanity.com/paper/2212.04593/) (2022)
- [Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoML](https://mlsanity.com/paper/2306.09297/) (2023)
- [Data vs. Model Machine Learning Fairness Testing: An Empirical Study](https://mlsanity.com/paper/2401.07697/) (2024)
- [Mind the Gap: Measuring Generalization Performance Across Multiple Objectives](https://mlsanity.com/paper/2212.04183/) (2022)
- [Navigating Ensemble Configurations for Algorithmic Fairness](https://mlsanity.com/paper/2210.05594/) (2022)
- [Function Composition in Trustworthy Machine Learning: Implementation Choices, Insights, and Questions](https://mlsanity.com/paper/2302.09190/) (2023)

## Abstract (arXiv)

There are several bias mitigators that can reduce algorithmic bias in machine learning models but, unfortunately, the effect of mitigators on fairness is often not stable when measured across different data splits. A popular approach to train more stable models is ensemble learning. Ensembles, such as bagging, boosting, voting, or stacking, have been successful at making predictive performance more stable. One might therefore ask whether we can combine the advantages of bias mitigators and ensembles? To explore this question, we first need bias mitigators and ensembles to work together. We built an open-source library enabling the modular composition of 10 mitigators, 4 ensembles, and their corresponding hyperparameters. Based on this library, we empirically explored the space of combinations on 13 datasets, including datasets commonly used in fairness literature plus datasets newly curated by our library. Furthermore, we distilled the results into a guidance diagram for practitioners. We hope this paper will contribute towards improving stability in bias mitigation.