# FairBatch: Batch Selection for Model Fairness

arXiv: https://arxiv.org/abs/2012.01696 · HTML: https://mlsanity.com/paper/2012.01696/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2020.

## Built on by (most cited first, 30 of 99)

- [LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Tasks](https://mlsanity.com/paper/2206.06565/) (2022)
- [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022)
- [FairFed: Enabling Group Fairness in Federated Learning](https://mlsanity.com/paper/2110.00857/) (2021)
- [Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective](https://mlsanity.com/paper/2112.06409/) (2021)
- [Sample Selection for Fair and Robust Training](https://mlsanity.com/paper/2110.14222/) (2021)
- [Improving Fairness via Federated Learning](https://mlsanity.com/paper/2110.15545/) (2021)
- [ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting](https://mlsanity.com/paper/2406.19976/) (2024)
- [GALAXY: Graph-based Active Learning at the Extreme](https://mlsanity.com/paper/2202.01402/) (2022)
- [On The Impact of Machine Learning Randomness on Group Fairness](https://mlsanity.com/paper/2307.04138/) (2023)
- [Optimising Equal Opportunity Fairness in Model Training](https://mlsanity.com/paper/2205.02393/) (2022)
- [BLIND: Bias Removal With No Demographics](https://mlsanity.com/paper/2212.10563/) (2022)
- [Re-weighting Based Group Fairness Regularization via Classwise Robust Optimization](https://mlsanity.com/paper/2303.00442/) (2023)
- [Anatomizing Bias in Facial Analysis](https://mlsanity.com/paper/2112.06522/) (2021)
- [Modeling Techniques for Machine Learning Fairness: A Survey](https://mlsanity.com/paper/2111.03015/) (2021)
- [On Fairness of Medical Image Classification with Multiple Sensitive Attributes via Learning Orthogonal Representations](https://mlsanity.com/paper/2301.01481/) (2023)
- [Improving Fairness in AI Models on Electronic Health Records: The Case for Federated Learning Methods](https://mlsanity.com/paper/2305.11386/) (2023)
- [Fairness through Aleatoric Uncertainty](https://mlsanity.com/paper/2304.03646/) (2023)
- [Improving Fair Training under Correlation Shifts](https://mlsanity.com/paper/2302.02323/) (2023)
- [SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning](https://mlsanity.com/paper/2305.19442/) (2023)
- [Equal Improvability: A New Fairness Notion Considering the Long-term Impact](https://mlsanity.com/paper/2210.06732/) (2022)
- [SYNAuG: Exploiting Synthetic Data for Data Imbalance Problems](https://mlsanity.com/paper/2308.00994/) (2023)
- [FORML: Learning to Reweight Data for Fairness](https://mlsanity.com/paper/2202.01719/) (2022)
- [Fair Federated Learning via Bounded Group Loss](https://mlsanity.com/paper/2203.10190/) (2022)
- [Input-agnostic Certified Group Fairness via Gaussian Parameter Smoothing](https://mlsanity.com/paper/2206.11423/) (2022)
- [Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP](https://mlsanity.com/paper/2302.05711/) (2023)
- [Loss Balancing for Fair Supervised Learning](https://mlsanity.com/paper/2311.03714/) (2023)
- [Same Same, But Different: Conditional Multi-Task Learning for Demographic-Specific Toxicity Detection](https://mlsanity.com/paper/2302.07372/) (2023)
- [Survey on Fairness Notions and Related Tensions](https://mlsanity.com/paper/2209.13012/) (2022)
- [FairGrad: Fairness Aware Gradient Descent](https://mlsanity.com/paper/2206.10923/) (2022)
- [FedDAR: Federated Domain-Aware Representation Learning](https://mlsanity.com/paper/2209.04007/) (2022)

## Abstract (arXiv)

Training a fair machine learning model is essential to prevent demographic disparity. Existing techniques for improving model fairness require broad changes in either data preprocessing or model training, rendering themselves difficult-to-adopt for potentially already complex machine learning systems. We address this problem via the lens of bilevel optimization. While keeping the standard training algorithm as an inner optimizer, we incorporate an outer optimizer so as to equip the inner problem with an additional functionality: Adaptively selecting minibatch sizes for the purpose of improving model fairness. Our batch selection algorithm, which we call FairBatch, implements this optimization and supports prominent fairness measures: equal opportunity, equalized odds, and demographic parity. FairBatch comes with a significant implementation benefit -- it does not require any modification to data preprocessing or model training. For instance, a single-line change of PyTorch code for replacing batch selection part of model training suffices to employ FairBatch. Our experiments conducted both on synthetic and benchmark real data demonstrate that FairBatch can provide such functionalities while achieving comparable (or even greater) performances against the state of the arts. Furthermore, FairBatch can readily improve fairness of any pre-trained model simply via fine-tuning. It is also compatible with existing batch selection techniques intended for different purposes, such as faster convergence, thus gracefully achieving multiple purposes.