mlsanityData to 7 Oct 2026

Fair Feature Distillation for Visual Recognition

arXiv 2106.04411 ↗First posted 27 May 2021cs.CV cs.AIcited by 59

Not on the map. Why?

Abstract (arXiv)

Fairness is becoming an increasingly crucial issue for computer vision, especially in the human-related decision systems. However, achieving algorithmic fairness, which makes a model produce indiscriminative outcomes against protected groups, is still an unresolved problem. In this paper, we devise a systematic approach which reduces algorithmic biases via feature distillation for visual recognition tasks, dubbed as MMD-based Fair Distillation (MFD). While the distillation technique has been widely used in general to improve the prediction accuracy, to the best of our knowledge, there has been no explicit work that also tries to improve fairness via distillation. Furthermore, We give a theoretical justification of our MFD on the effect of knowledge distillation and fairness. Throughout the extensive experiments, we show our MFD significantly mitigates the bias against specific minorities without any loss of the accuracy on both synthetic and real-world face datasets.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Builds on 25 of its 45 references that are in the map. First posted on arXiv 27 May 2021.

Builds on

25 of 45 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  6. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  7. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  8. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  9. 2019Relational Knowledge Distillationcited by 873
  10. 2017Counterfactual Fairnesscited by 802
  11. 2019Contrastive Representation Distillationcited by 734
  12. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
13 more
  1. 2018A Reductions Approach to Fair Classificationcited by 585
  2. 2018Born Again Neural Networkscited by 539
  3. 2017Age Progression/Regression by Conditional Adversarial Autoencodercited by 426
  4. 2019Variational Information Distillation for Knowledge Transfercited by 321
  5. 2018AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Biascited by 319
  6. 2017Fairness in Criminal Justice Risk Assessments: The State of the Artcited by 315
  7. 2018Deep Face Recognition: A Surveycited by 275
  8. 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
  9. 2015The Variational Fair Autoencodercited by 223
  10. 2018Learning Deep Representations with Probabilistic Knowledge Transfercited by 209
  11. 2019Flexibly Fair Representation Learning by Disentanglementcited by 196
  12. 2019Identifying and Correcting Label Bias in Machine Learningcited by 146
  13. 2019Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learningcited by 145

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 10; 2023: 18; 2024: 17; 2025: 3; 2026: 5.

59 papers.

Sort:

By area:

  1. 2022Fair Contrastive Learning for Facial Attribute Classificationcited by 71
  2. 2023Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifierscited by 66
  3. 2021Learning Fair Classifiers with Partially Annotated Group Labelscited by 40
  4. 2022On Learning Fairness and Accuracy on Multiple Subgroupscited by 31
  5. 2021Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learningcited by 25
  6. 2024FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSscited by 25
  7. 2022Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Surveycited by 22
  8. 2023Re-weighting Based Group Fairness Regularization via Classwise Robust Optimizationcited by 20
  9. 2021Fast and Efficient MMD-based Fair PCA via Optimization over Stiefel Manifoldcited by 18
  10. 2021Anatomizing Bias in Facial Analysiscited by 18
  11. 2023Resilient Constrained Learningcited by 15
  12. 2023Toward Fairness Through Fair Multi-Exit Framework for Dermatological Disease Diagnosiscited by 14
38 more
  1. 2023Semantic Scene Completion with Cleaner Selfcited by 12
  2. 2024Fairness and Bias Mitigation in Computer Vision: A Surveycited by 12
  3. 2024Achieve Fairness without Demographics for Dermatological Disease Diagnosiscited by 10
  4. 2023Racial Bias within Face Recognition: A Surveycited by 9
  5. 2024Achieving Fairness Through Channel Pruning for Dermatological Disease Diagnosiscited by 9
  6. 2022CoupleFace: Relation Matters for Face Recognition Distillationcited by 8
  7. 2023A Multidimensional Analysis of Social Biases in Vision Transformerscited by 8
  8. 2021Distill and De-bias: Mitigating Bias in Face Verification using Knowledge Distillationcited by 6
  9. 2023Fairness-aware Vision Transformer via Debiased Self-Attentioncited by 6
  10. 2024A Large-Scale Empirical Study on Improving the Fairness of Image Classification Modelscited by 6
  11. 2025Incorporating Rather Than Eliminating: Achieving Fairness for Skin Disease Diagnosis Through Group-Specific Expertcited by 5
  12. 2023Fair Multi-Exit Framework for Facial Attribute Classificationcited by 4
  13. 2024What is Left After Distillation? How Knowledge Transfer Impacts Fairness and Biascited by 4
  14. 2022Enhancing Fairness of Visual Attribute Predictorscited by 3
  15. 2023Backdoor for Debias: Mitigating Model Bias with Backdoor Attack-based Artificial Biascited by 3
  16. 2023Boosting Residual Networks with Group Knowledgecited by 3
  17. 2023LumiNet: Perception-Driven Knowledge Distillation via Statistical Logit Calibrationcited by 3
  18. 2024FairDD: Fair Dataset Distillationcited by 3
  19. 2023To be Robust and to be Fair: Aligning Fairness with Robustnesscited by 2
  20. 2023Benign Shortcut for Debiasing: Fair Visual Recognition via Intervention with Shortcut Featurescited by 2
  21. 2023Dual-Teacher De-biasing Distillation Framework for Multi-domain Fake News Detectioncited by 2
  22. 2024Biasing & Debiasing based Approach Towards Fair Knowledge Transfer for Equitable Skin Analysiscited by 2
  23. 2025Controllable Feature Whitening for Hyperparameter-Free Bias Mitigationcited by 2
  24. 2022Debiasing Backdoor Attack: A Benign Application of Backdoor Attack in Eliminating Data Biascited by 1
  25. 2022A Differentiable Distance Approximation for Fairer Image Classificationcited by 1
  26. 2022Mitigating Urban-Rural Disparities in Contrastive Representation Learning with Satellite Imagerycited by 1
  27. 2023Continual Learning in the Presence of Spurious Correlationcited by 1
  28. 2024An Empirical Study of the Influence of Adversarial Fine-Tuning on Compressed Neural Networkscited by 1
  29. 2024Approximating Discrimination Within Models When Faced With Several Non-Binary Sensitive Attributescited by 1
  30. 2024Generating Synthetic Fair Syntax-agnostic Data by Learning and Distilling Fair Representationcited by 1
  31. 2024How Knowledge Distillation Mitigates the Synthetic Gap in Fair Face Recognitioncited by 1
  32. 2024Will the Inclusion of Generated Data Amplify Bias Across Generations in Future Image Classification Models?cited by 1
  33. 2026SHaSaM: Submodular Hard Sample Mining for Fair Facial Attribute Recognitioncited by 1
  34. 2021F3: Fair and Federated Face Attribute Classification with Heterogeneous Datacited by 0
  35. 2022Disparate Conditional Prediction in Multiclass Classifierscited by 0
  36. 2022Fair Visual Recognition via Intervention with Proxy Featurescited by 0
  37. 2023Fairness meets Cross-Domain Learning: a new perspective on Models and Metricscited by 0
  38. 2023StableKD: Breaking Inter-block Optimization Entanglement for Stable Knowledge Distillationcited by 0

50 most cited of 59 All 59 (JSON)