FeDa4Fair: Client-Level Federated Datasets for Fairness Evaluation
Not on the map. Why?
Abstract (arXiv)
Federated Learning (FL) enables collaborative training while preserving privacy, yet it introduces a critical challenge: the "illusion of fairness''. A global model, usually evaluated on the server, appears fair on average while keeping persistent discrimination at the client level. Current fairness-enhancing FL solutions often fall short, as they typically mitigate biases for a single, usually binary, sensitive attribute, while ignoring two realistic and conflicting scenarios: attribute-bias (where clients are unfair toward different sensitive attributes) and value-bias (where clients exhibit conflicting biases toward different values of the same attribute). To support more robust and reproducible fairness research in FL, we introduce FeDa4Fair, the first benchmarking framework designed to stress-test fairness methods under these heterogeneous conditions. Our contributions are three-fold: (1) We introduce FeDa4Fair, a library designed to create datasets tailored to evaluating fair FL methods under heterogeneous client bias; (2) we release a benchmark suite generated by the FeDa4Fair library to standardize the evaluation of fair FL methods; (3) we provide ready-to-use functions for evaluating fairness outcomes for these datasets.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 63 references that are in the map. First posted on arXiv 26 Jun 2025.
Builds on
32 of 63 references.
References · 32
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
- 2014Deep Learning Face Attributes in the Wildcited by 3,693
- 2016Equality of Opportunity in Supervised Learningcited by 2,007
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
- 2018LEAF: A Benchmark for Federated Settingscited by 792
- 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 672
- 2021Federated Learning on Non-IID Data Silos: An Experimental Studycited by 568
- 2021Datasets: A Community Library for Natural Language Processingcited by 530
- 2019Agnostic Federated Learningcited by 456
- 2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
- 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 449
20 more
- 2019Fair Resource Allocation in Federated Learningcited by 394
- 2020Fairness in Machine Learning: A Surveycited by 370
- 2021Federated Learning on Non-IID Data: A Surveycited by 346
- 2020Fairness in Machine Learningcited by 194
- 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
- 2020Salvaging Federated Learning by Local Adaptationcited by 142
- 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
- 2020When Machine Learning Meets Privacy: A Survey and Outlookcited by 68
- 2021Federated Learning with Fair Averagingcited by 61
- 2020Poisoning Attacks on Algorithmic Fairnesscited by 59
- 2020Mitigating Bias in Federated Learningcited by 56
- 2021Enforcing fairness in private federated learning via the modified method of differential multiplierscited by 38
- 2022Minimax Demographic Group Fairness in Federated Learningcited by 24
- 2022PrivFairFL: Privacy-Preserving Group Fairness in Federated Learningcited by 13
- 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 13
- 2023Bias Propagation in Federated Learningcited by 12
- 2023Mitigating Group Bias in Federated Learning for Heterogeneous Devicescited by 5
- 2024PUFFLE: Balancing Privacy, Utility, and Fairness in Federated Learningcited by 4
- 2025Global Group Fairness in Federated Learning via Function Trackingcited by 1
- 2025Fairness in Federated Learning: Fairness for Whom?cited by 1
Built on by
Not cited yet.