# Fairness via In-Processing in the Over-parameterized Regime: A Cautionary Tale

arXiv: https://arxiv.org/abs/2206.14853 · HTML: https://mlsanity.com/paper/2206.14853/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 63 references that are in the map.
- First posted on arXiv 29 Jun 2022.

## Builds on (most cited first, 30 of 36)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [An Investigation of Why Overparameterization Exacerbates Spurious Correlations](https://mlsanity.com/paper/2005.04345/) (2020)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [Empirical Risk Minimization under Fairness Constraints](https://mlsanity.com/paper/1802.08626/) (2018)
- [Fairness of Exposure in Rankings](https://mlsanity.com/paper/1802.07281/) (2018)
- [On the Origin of Implicit Regularization in Stochastic Gradient Descent](https://mlsanity.com/paper/2101.12176/) (2021)
- [Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations](https://mlsanity.com/paper/1707.00075/) (2017)
- [Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks](https://mlsanity.com/paper/1903.11680/) (2019)
- [Minimax Pareto Fairness: A Multi Objective Perspective](https://mlsanity.com/paper/2011.01821/) (2020)
- [Implicit Gradient Regularization](https://mlsanity.com/paper/2009.11162/) (2020)
- [Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learning](https://mlsanity.com/paper/1911.10692/) (2019)
- [Disparities in Dermatology AI: Assessments Using Diverse Clinical Images](https://mlsanity.com/paper/2111.08006/) (2021)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [Equity of Attention: Amortizing Individual Fairness in Rankings](https://mlsanity.com/paper/1805.01788/) (2018)
- [Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction](https://mlsanity.com/paper/1807.00199/) (2018)
- [Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements](https://mlsanity.com/paper/1901.04562/) (2019)
- [Do We Need Zero Training Loss After Achieving Zero Training Error?](https://mlsanity.com/paper/2002.08709/) (2020)
- [Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classification](https://mlsanity.com/paper/1906.05082/) (2019)
- [FNNC: Achieving Fairness through Neural Networks](https://mlsanity.com/paper/1811.00247/) (2018)
- [Toward a better trade-off between performance and fairness with kernel-based distribution matching](https://mlsanity.com/paper/1910.11779/) (2019)

## Built on by (most cited first, 4 of 4)

- [FRAPPE: A Group Fairness Framework for Post-Processing Everything](https://mlsanity.com/paper/2312.02592/) (2023)
- [Malign Overfitting: Interpolation Can Provably Preclude Invariance](https://mlsanity.com/paper/2211.15724/) (2022)
- [SoK: Unintended Interactions among Machine Learning Defenses and Risks](https://mlsanity.com/paper/2312.04542/) (2023)
- [Hyper-parameter Tuning for Fair Classification without Sensitive Attribute Access](https://mlsanity.com/paper/2302.01385/) (2023)

## Abstract (arXiv)

The success of DNNs is driven by the counter-intuitive ability of over-parameterized networks to generalize, even when they perfectly fit the training data. In practice, test error often continues to decrease with increasing over-parameterization, referred to as double descent. This allows practitioners to instantiate large models without having to worry about over-fitting. Despite its benefits, however, prior work has shown that over-parameterization can exacerbate bias against minority subgroups. Several fairness-constrained DNN training methods have been proposed to address this concern. Here, we critically examine MinDiff, a fairness-constrained training procedure implemented within TensorFlow's Responsible AI Toolkit, that aims to achieve Equality of Opportunity. We show that although MinDiff improves fairness for under-parameterized models, it is likely to be ineffective in the over-parameterized regime. This is because an overfit model with zero training loss is trivially group-wise fair on training data, creating an "illusion of fairness," thus turning off the MinDiff optimization (this will apply to any disparity-based measures which care about errors or accuracy. It won't apply to demographic parity). Within specified fairness constraints, under-parameterized MinDiff models can even have lower error compared to their over-parameterized counterparts (despite baseline over-parameterized models having lower error). We further show that MinDiff optimization is very sensitive to choice of batch size in the under-parameterized regime. Thus, fair model training using MinDiff requires time-consuming hyper-parameter searches. Finally, we suggest using previously proposed regularization techniques, viz. L2, early stopping and flooding in conjunction with MinDiff to train fair over-parameterized models.