mlsanityData to 7 Oct 2026

Leave-one-out Unfairness

arXiv 2107.10171 ↗First posted 21 Jul 2021cs.LG cs.CYcited by 36

Not on the map. Why?

Abstract (arXiv)

We introduce leave-one-out unfairness, which characterizes how likely a model's prediction for an individual will change due to the inclusion or removal of a single other person in the model's training data. Leave-one-out unfairness appeals to the idea that fair decisions are not arbitrary: they should not be based on the chance event of any one person's inclusion in the training data. Leave-one-out unfairness is closely related to algorithmic stability, but it focuses on the consistency of an individual point's prediction outcome over unit changes to the training data, rather than the error of the model in aggregate. Beyond formalizing leave-one-out unfairness, we characterize the extent to which deep models behave leave-one-out unfairly on real data, including in cases where the generalization error is small. Further, we demonstrate that adversarial training and randomized smoothing techniques have opposite effects on leave-one-out fairness, which sheds light on the relationships between robustness, memorization, individual fairness, and leave-one-out fairness in deep models. Finally, we discuss salient practical applications that may be negatively affected by leave-one-out unfairness.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 33 of its 67 references that are in the map. First posted on arXiv 21 Jul 2021.

Builds on

33 of 67 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  6. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  8. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  9. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  10. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  11. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  12. 2017A Closer Look at Memorization in Deep Networkscited by 936
21 more
  1. 2018Robustness May Be at Odds with Accuracycited by 873
  2. 2017Counterfactual Fairnesscited by 802
  3. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  4. 2016The Mythos of Model Interpretabilitycited by 716
  5. 2014Certifying and removing disparate impactcited by 715
  6. 2017Renyi Differential Privacycited by 643
  7. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  8. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 547
  9. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  10. 2019Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practicescited by 170
  11. 2019Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inferencecited by 168
  12. 2016Fairness in Learning: Classic and Contextual Banditscited by 167
  13. 2017Machine Learning Models that Remember Too Muchcited by 165
  14. 2014Preserving Statistical Validity in Adaptive Data Analysiscited by 102
  15. 2017Data-Dependent Stability of Stochastic Gradient Descentcited by 90
  16. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  17. 2018Fairness Under Compositioncited by 43
  18. 2019E.T.-RNN: Applying Deep Learning to Credit Loan Applicationscited by 26
  19. 2020Individual Fairness Revisited: Transferring Techniques from Adversarial Robustnesscited by 17
  20. 2017Use Privacy in Data-Driven Systems: Theory and Experiments with Machine Learnt Programscited by 7
  21. 2019An Empirical Study on the Relation between Network Interpretability and Adversarial Robustnesscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 9; 2024: 5; 2025: 10; 2026: 3.

36 papers.

Sort:

  1. 2022Training Data Influence Analysis and Estimation: A Surveycited by 134
  2. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  3. 2022Predictive Multiplicity in Probabilistic Classificationcited by 44
  4. 2021Consistent Counterfactuals for Deep Modelscited by 37
  5. 2022Understanding Instance-Level Impact of Fairness Constraintscited by 27
  6. 2021Selective Ensembles for Consistent Predictionscited by 23
  7. 2021Certifying Robustness to Programmable Data Bias in Decision Treescited by 21
  8. 2022Rank List Sensitivity of Recommender Systems to Interaction Perturbationscited by 20
  9. 2023The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictionscited by 17
  10. 2022Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Treescited by 16
  11. 2024Predictive Churn with the Set of Good Modelscited by 15
  12. 2023Multi-Target Multiplicity: Flexibility and Fairness in Target Specification under Resource Constraintscited by 11
24 more
  1. 2025Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Setcited by 10
  2. 2024Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspectivecited by 9
  3. 2023Towards Algorithmic Fairness by means of Instance-level Data Re-weighting based on Shapley Valuescited by 7
  4. 2023Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Toolscited by 7
  5. 2023Fairness and Sequential Decision Making: Limits, Lessons, and Opportunitiescited by 5
  6. 2023A Simple and Efficient Baseline for Data Attribution on Imagescited by 5
  7. 2024Towards Effective Discrimination Testing for Generative AIcited by 5
  8. 2022Certifying Data-Bias Robustness in Linear Regressioncited by 3
  9. 2023On Minimizing the Impact of Dataset Shifts on Actionable Explanationscited by 3
  10. 2024Different Horses for Different Courses: Comparing Bias Mitigation Algorithms in MLcited by 3
  11. 2024FINEST: Stabilizing Recommendations by Rank-Preserving Fine-Tuningcited by 2
  12. 2025Fast-DataShapley: Neural Modeling for Training Data Valuationcited by 2
  13. 2025Testing Most Influential Setscited by 2
  14. 2023An Epistemic and Aleatoric Decomposition of Arbitrariness to Constrain the Set of Good Modelscited by 1
  15. 2025Enhancing Interpretability in Generative AI Through Search-Based Data Influence Analysiscited by 1
  16. 2025Detect & Score: Privacy-Preserving Misbehaviour Detection and Contribution Evaluation in Federated Learningcited by 1
  17. 2025Training Data Attribution for Image Generation using Ontology-Aligned Knowledge Graphscited by 1
  18. 2026Analyzing Fairness of Neural Network Prediction via Counterfactual Dataset Generationcited by 1
  19. 2026Private and Robust Contribution Evaluation in Federated Learningcited by 1
  20. 2025A case for data valuation transparency via DValCardscited by 0
  21. 2025On Arbitrary Predictions from Equally Valid Modelscited by 0
  22. 2025Empowering Denoising Sequential Recommendation with Large Language Model Embeddingscited by 0
  23. 2025Revisiting the Relation Between Robustness and Universalitycited by 0
  24. 2026FrED: External Data Influence Estimation via Domain Knowledge Graph Groundingcited by 0