mlsanityData to 7 Oct 2026

On The Impact of Machine Learning Randomness on Group Fairness

arXiv 2307.04138 ↗First posted 9 Jul 2023cs.LG cs.CYcited by 26

Not on the map. Why?

Abstract (arXiv)

Statistical measures for group fairness in machine learning reflect the gap in performance of algorithms across different groups. These measures, however, exhibit a high variance between different training instances, which makes them unreliable for empirical evaluation of fairness. What causes this high variance? We investigate the impact on group fairness of different sources of randomness in training neural networks. We show that the variance in group fairness measures is rooted in the high volatility of the learning process on under-represented groups. Further, we recognize the dominant source of randomness as the stochasticity of data order during training. Based on these findings, we show how one can control group-level accuracy (i.e., model fairness), with high efficiency and negligible impact on the model's overall performance, by simply changing the data order for a single epoch.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 36 of its 64 references that are in the map. First posted on arXiv 9 Jul 2023.

Builds on

36 of 64 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  6. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  7. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  8. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  9. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 460
  10. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 443
  11. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  12. 2020Fairness in Machine Learning: A Surveycited by 369
24 more
  1. 2018Essentially No Barriers in Neural Network Energy Landscapecited by 328
  2. 2018AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Biascited by 319
  3. 2021Curriculum Learning: A Surveycited by 257
  4. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  5. 2018A comparative study of fairness-enhancing interventions in machine learningcited by 208
  6. 2020Characterising Bias in Compressed Modelscited by 162
  7. 2018The Frontiers of Fairness in Machine Learningcited by 139
  8. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  9. 2020Random Reshuffling: Simple Analysis with Vast Improvementscited by 112
  10. 2020FairBatch: Batch Selection for Model Fairnesscited by 99
  11. 2013Dropout Training as Adaptive Regularizationcited by 89
  12. 2019BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performancecited by 89
  13. 2021The MultiBERTs: BERT Reproductions for Robustness Analysiscited by 89
  14. 2021Manipulating SGD with Data Ordering Attackscited by 73
  15. 2020A Unified Convergence Analysis for Shuffling-Type Gradient Methodscited by 57
  16. 2017Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimizationcited by 56
  17. 2021Your fairness may vary: Pretrained language model fairness in toxic text classificationcited by 38
  18. 2019Fairness in representation: quantifying stereotyping as a representational harmcited by 26
  19. 2020Choosing the Sample with Lowest Loss makes SGD Robustcited by 26
  20. 2022GraB: Finding Provably Better Data Permutations than Random Reshufflingcited by 18
  21. 2021Permutation-Based SGD: Is Random Optimal?cited by 16
  22. 2022Characterizing & Finding Good Data Orderings for Fast Convergence of Sequential Gradient Methodscited by 15
  23. 2022We need to talk about random seedscited by 12
  24. 2021On the Impact of Random Seeds on the Fairness of Clinical Classifierscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 12; 2025: 7; 2026: 3.

26 papers.

Sort:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  3. 2024LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actionscited by 21
  4. 2025Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learningcited by 17
  5. 2024Predictive Churn with the Set of Good Modelscited by 15
  6. 2024Optimal Protocols for Continual Learning via Statistical Physics and Control Theorycited by 14
  7. 2025Assessing the Macro and Micro Effects of Random Seeds on Fine-Tuning Large Language Modelscited by 10
  8. 2024Tilting the Odds at the Lottery: the Interplay of Overparameterisation and Curricula in Neural Networkscited by 9
  9. 2023Understanding Fairness Surrogate Functions in Algorithmic Fairnesscited by 8
  10. 2024Bias in Motion: Theoretical Insights into the Dynamics of Bias in SGD Trainingcited by 8
  11. 2023An Empirical Investigation into Benchmarking Model Multiplicity for Trustworthy Machine Learning: A Case Study on Image Classificationcited by 7
  12. 2024Towards Effective Discrimination Testing for Generative AIcited by 5
14 more
  1. 2024Stability and Multigroup Fairness in Ranking with Uncertain Predictionscited by 4
  2. 2024Position: Cracking the Code of Cascading Disparity Towards Marginalized Communitiescited by 4
  3. 2024The Cost of Arbitrariness for Individuals: Examining the Legal and Technical Challenges of Model Multiplicitycited by 4
  4. 2025Fairness and Sparsity within Rashomon sets: Enumeration-Free Exploration and Characterizationcited by 4
  5. 2024Different Horses for Different Courses: Comparing Bias Mitigation Algorithms in MLcited by 3
  6. 2024Fragile Giants: Understanding the Susceptibility of Models to Subpopulation Attackscited by 1
  7. 2025Fairness Dynamics During Trainingcited by 1
  8. 2024(Un)certainty of (Un)fairness: Preference-Based Selection of Certainly Fair Decision-Makerscited by 0
  9. 2025A Critical Review of Predominant Bias in Neural Networkscited by 0
  10. 2025On Arbitrary Predictions from Equally Valid Modelscited by 0
  11. 2025Unequal Uncertainty: Rethinking Algorithmic Interventions for Mitigating Discrimination from AIcited by 0
  12. 2026One RNG to Rule Them All: How Randomness Becomes an Attack Vector in Machine Learningcited by 0
  13. 2026Temporal Taxation Compounds Under Post-Training Compression of Whisper Modelscited by 0
  14. 2026Fairness Beyond a Single Run: Training-Seed Variability in Speech LLM Adaptationcited by 0