# Improving Fairness with Ensemble Combination: Margin-Dependent Bounds

arXiv: https://arxiv.org/abs/2301.10813 · HTML: https://mlsanity.com/paper/2301.10813/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 70 references that are in the map.
- First posted on arXiv 25 Jan 2023.

## Builds on (most cited first, 27 of 27)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [The Variational Fair Autoencoder](https://mlsanity.com/paper/1511.00830/) (2015)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Fair Clustering Through Fairlets](https://mlsanity.com/paper/1802.05733/) (2018)
- [Fairness in Learning: Classic and Contextual Bandits](https://mlsanity.com/paper/1605.07139/) (2016)
- [Fair Regression: Quantitative Definitions and Reduction-based Algorithms](https://mlsanity.com/paper/1905.12843/) (2019)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [Scalable Fair Clustering](https://mlsanity.com/paper/1902.03519/) (2019)
- [The Price of Fair PCA: One Extra Dimension](https://mlsanity.com/paper/1811.00103/) (2018)
- [On Formalizing Fairness in Prediction with Machine Learning](https://mlsanity.com/paper/1710.03184/) (2017)
- [AdaFair: Cumulative Fairness Adaptive Boosting](https://mlsanity.com/paper/1909.08982/) (2019)
- [Fairness Under Composition](https://mlsanity.com/paper/1806.06122/) (2018)
- [Causal Conceptions of Fairness and their Consequences](https://mlsanity.com/paper/2207.05302/) (2022)
- [FARF: A Fair and Adaptive Random Forests Classifier](https://mlsanity.com/paper/2108.07403/) (2021)
- [Second Order PAC-Bayesian Bounds for the Weighted Majority Vote](https://mlsanity.com/paper/2007.13532/) (2020)
- [FairGBM: Gradient Boosting with Fairness Constraints](https://mlsanity.com/paper/2209.07850/) (2022)
- [On Fairness, Diversity and Randomness in Algorithmic Decision Making](https://mlsanity.com/paper/1706.10208/) (2017)
- [Ensemble Pruning based on Objection Maximization with a General Distributed Framework](https://mlsanity.com/paper/1806.04899/) (2018)

## Built on by (most cited first, 3 of 3)

- [Measuring Model-Induced Discrimination via Efficient Fairness Approximation](https://mlsanity.com/paper/2405.09251/) (2024)
- [FairSHAP: Preprocessing for Fairness Through Attribution-Based Data Augmentation](https://mlsanity.com/paper/2505.11111/) (2025)
- [Algorithmic Fairness: Not a Purely Technical but Socio-Technical Property](https://mlsanity.com/paper/2506.12556/) (2025)

## Abstract (arXiv)

The concern about hidden discrimination in machine learning models is growing, as their widespread real-world applications increasingly impact human lives. Various techniques, including commonly used group fairness measures and several fairness-aware ensemble-based methods, have been developed to enhance fairness. However, existing fairness measures typically focus on only one aspect -- either group or individual fairness, and the compatibility difficulty among these measures indicates a possibility of remaining biases even when one of them is satisfied. Moreover, existing mechanisms to boost fairness usually present empirical results to show validity, yet few of them discuss whether fairness can be boosted with certain theoretical guarantees. To address these issues, we propose a fairness quality measure named `discriminative risk' by only perturbing protected attributes in instances, to express both individual and group fairness aspects. Furthermore, we investigate its properties and establish the first- and second-order oracle bounds and their relaxations, which show that fairness is possibly improved via ensemble combination with margin-dependent bounds. The analysis is suitable for both binary and multi-class classification. A few ensemble pruning methods are also proposed to utilise our proposed measure and obtain both accurate and fair sub-ensembles; comprehensive experiments are conducted to evaluate the effectiveness of the proposed fairness measure and pruning methods.