SoK: Taming the Triangle -- On the Interplays between Fairness, Interpretability and Privacy in Machine Learning
Not on the map. Why?
Abstract (arXiv)
Machine learning techniques are increasingly used for high-stakes decision-making, such as college admissions, loan attribution or recidivism prediction. Thus, it is crucial to ensure that the models learnt can be audited or understood by human users, do not create or reproduce discrimination or bias, and do not leak sensitive information regarding their training data. Indeed, interpretability, fairness and privacy are key requirements for the development of responsible machine learning, and all three have been studied extensively during the last decade. However, they were mainly considered in isolation, while in practice they interplay with each other, either positively or negatively. In this Systematization of Knowledge (SoK) paper, we survey the literature on the interactions between these three desiderata. More precisely, for each pairwise interaction, we summarize the identified synergies and tensions. These findings highlight several fundamental theoretical and empirical conflicts, while also demonstrating that jointly considering these different requirements is challenging when one aims at preserving a high level of utility. To solve this issue, we also discuss possible conciliation mechanisms, showing that a careful design can enable to successfully handle these different concerns in practice.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 77 of its 134 references that are in the map. First posted on arXiv 22 Dec 2023.
Builds on
77 of 134 references.
References · 77
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2016Deep Learning with Differential Privacycited by 2,728
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2016Equality of Opportunity in Supervised Learningcited by 2,007
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
- 2017SmoothGrad: removing noise by adding noisecited by 1,109
- 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
- 2016The Mythos of Model Interpretabilitycited by 721
38 more
- 2016Stealing Machine Learning Models via Prediction APIscited by 619
- 2018A Reductions Approach to Fair Classificationcited by 587
- 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
- 2017On Fairness and Calibrationcited by 406
- 2020Fairness in Machine Learning: A Surveycited by 370
- 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 367
- 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
- 2018Actionable Recourse in Linear Classificationcited by 328
- 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
- 2018Scalable Private Learning with PATEcited by 269
- 2018A comparative study of fairness-enhancing interventions in machine learningcited by 208
- 2020Problems with Shapley-value-based explanations as feature importance measurescited by 200
- 2017Avoiding Discrimination through Causal Reasoningcited by 194
- 2020A Survey of Privacy Attacks in Machine Learningcited by 146
- 2021Counterfactual Explanations Can Be Manipulatedcited by 113
- 2019Fooling Neural Network Interpretations via Adversarial Model Manipulationcited by 112
- 2019Learning to Deceive with Attention-Based Explanationscited by 111
- 2019"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanationscited by 111
- 2018Model Reconstruction from Model Explanationscited by 86
- 2020On the Privacy Risks of Algorithmic Fairnesscited by 82
- 2019Fairwashing: the risk of rationalizationcited by 81
- 2018Differentially Private Fair Learningcited by 76
- 2014Differential Privacy and Machine Learning: a Survey and Reviewcited by 67
- 2020Neither Private Nor Fair: Impact of Data Imbalance on Utility and Fairness in Differential Privacycited by 61
- 2019Fair Decision Making using Privacy-Protected Datacited by 58
- 2019Equalizing Recourse across Groupscited by 55
- 2020Differentially Private and Fair Deep Learning: A Lagrangian Dual Approachcited by 53
- 2021Exploiting Explanations for Model Inversion Attackscited by 49
- 2020Model extraction from counterfactual explanationscited by 47
- 2022Differential Privacy and Fairness in Decisions and Learning Tasks: A Surveycited by 47
- 2020Fair Learning with Private Demographic Datacited by 46
- 2020In Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism Predictioncited by 42
- 2018Blind Justice: Fairness with Encrypted Sensitive Attributescited by 41
- 2022The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanationscited by 41
- 2022Fairness via Explanation Quality: Evaluating Disparities in the Quality of Post hoc Explanationscited by 40
- 2020An Overview of Privacy in Machine Learningcited by 39
- 2020Chasing Your Long Tails: Differentially Private Prediction in Health Care Settingscited by 37
- 2019Disparate Vulnerability to Membership Inference Attackscited by 35
50 most cited of 77 All 77 (JSON)
Built on by
Citing papers per year: 2024: 2; 2025: 2.
4 papers.
- 2025Crowding Out The Noise: Algorithmic Collective Action Under Differential Privacycited by 4
- 2024Rolling in the deep of cognitive and AI biasescited by 2
- 2024Combining Machine Learning Defenses without Conflictscited by 2
- 2025Tabular Diffusion based Actionable Counterfactual Explanations for Network Intrusion Detectioncited by 1
- 2025Tabular Diffusion based Actionable Counterfactual Explanations for Network Intrusion Detection23 Jul 2025
- 2025Crowding Out The Noise: Algorithmic Collective Action Under Differential Privacy9 May 2025
- 2024Combining Machine Learning Defenses without Conflicts14 Nov 2024
- 2024Rolling in the deep of cognitive and AI biases30 Jul 2024