mlsanityData to 13 Oct 2026

Last-Layer Fairness Fine-tuning is Simple and Effective for Neural Networks

arXiv 2304.03935 ↗First posted 8 Apr 2023cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

As machine learning has been deployed ubiquitously across applications in modern data science, algorithmic fairness has become a great concern. Among them, imposing fairness constraints during learning, i.e. in-processing fair training, has been a popular type of training method because they don't require accessing sensitive attributes during test time in contrast to post-processing methods. While this has been extensively studied in classical machine learning models, their impact on deep neural networks remains unclear. Recent research has shown that adding fairness constraints to the objective function leads to severe over-fitting to fairness criteria in large models, and how to solve this challenge is an important open question. To tackle this, we leverage the wisdom and power of pre-training and fine-tuning and develop a simple but novel framework to train fair neural networks in an efficient and inexpensive way -- last-layer fine-tuning alone can effectively promote fairness in deep neural networks. This framework offers valuable insights into representation learning for training fair neural networks.

Key facts. Cited by 17 papers in the map (data through 13 Oct 2026). Builds on 29 of its 40 references that are in the map. First posted on arXiv 8 Apr 2023.

Builds on

29 of 40 references.

References · 29

  1. 2014Deep Learning Face Attributes in the Wildcited by 3,686
  2. 2016Equality of Opportunity in Supervised Learningcited by 2,005
  3. 2019A Comprehensive Survey on Transfer Learningcited by 1,157
  4. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
  5. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  6. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 444
  7. 2017Age Progression/Regression by Conditional Adversarial Autoencodercited by 428
  8. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 380
  9. 2018Learning Adversarially Fair and Transferable Representationscited by 353
  10. 2020Fairness without Demographics through Adversarially Reweighted Learningcited by 251
  11. 2018Multiaccuracy: Black-Box Post-Processing for Fairness in Classificationcited by 221
  12. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
17 more
  1. 2020How Does Mixup Help With Robustness and Generalization?cited by 193
  2. 2019On the Apparent Conflict Between Individual and Group Fairnesscited by 126
  3. 2021An Unconstrained Layer-Peeled Perspective on Neural Collapsecited by 87
  4. 2021Post-processing for Individual Fairnesscited by 61
  5. 2021The Power of Contrast for Feature Learning: A Theoretical Analysiscited by 53
  6. 2018FNNC: Achieving Fairness through Neural Networkscited by 43
  7. 2020Toward Better Generalization Bounds with Locally Elastic Stabilitycited by 42
  8. 2021Adversarial Training Helps Transfer Learning via Better Representationscited by 36
  9. 2020Representation via Representations: Domain Generalization via Adversarially Learned Invariant Representationscited by 26
  10. 2020Improving Adversarial Robustness via Unlabeled Out-of-Domain Datacited by 23
  11. 2022Robustness Implies Generalization via Data-Dependent Generalization Boundscited by 23
  12. 2021Technical Challenges for Training Fair Neural Networkscited by 20
  13. 2022FIFA: Making Fairness More Generalizable in Classifiers Trained on Imbalanced Datacited by 13
  14. 2021Understanding Dynamics of Nonlinear Representation Learning and Its Applicationcited by 11
  15. 2020README: REpresentation learning by fairness-Aware Disentangling MEthodcited by 10
  16. 2022Reinforcement Learning with Stepwise Fairness Constraintscited by 9
  17. 2021Scaffolding Setscited by 8

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 1; 2024: 11; 2025: 4; 2026: 1.

17 papers.

Sort:

By area:

  1. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Modelscited by 64
  2. 2025Detecting High-Stakes Interactions with Activation Probescited by 50
  3. 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Drivingcited by 35
  4. 2024Multilinear Mixture of Experts: Scalable Expert Specialization through Factorizationcited by 28
  5. 2024When is Multicalibration Post-Processing Necessary?cited by 16
  6. 2024Mind the Graph When Balancing Data for Fairness or Robustnesscited by 9
  7. 2024A Large-Scale Empirical Study on Improving the Fairness of Image Classification Modelscited by 6
  8. 2024BMFT: Achieving Fairness via Bias-based Weight Masking Fine-tuningcited by 6
  9. 2024Using Self-supervised Learning Can Improve Model Fairnesscited by 5
  10. 2024Position: Cracking the Code of Cascading Disparity Towards Marginalized Communitiescited by 4
  11. 2024Looking Beyond What You See: An Empirical Analysis on Subgroup Intersectional Fairness for Multi-label Chest X-ray Classification Using Social Determinants of Racial Health Inequitiescited by 3
  12. 2025AnyPcc: Compressing Any Point Cloud with a Single Universal Modelcited by 1
5 more
  1. 2023The Fairness Stitch: Unveiling the Potential of Model Stitching in Neural Network De-Biasingcited by 0
  2. 2024Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotationcited by 0
  3. 2025Fairness-Aware Low-Rank Representation Fine-Tuningcited by 0
  4. 2025SWiFT: Soft-Mask Weight Fine-tuning for Bias Mitigationcited by 0
  5. 2026Beyond Demographic Balance: Multi-Metric and Intersectional Evaluation of Fairness in MIMIC-IV Mortality Predictioncited by 0