mlsanityData to 7 Oct 2026

Antipodes of Label Differential Privacy: PATE and ALIBI

arXiv 2106.03408 ↗First posted 7 Jun 2021cs.LG cs.CRcited by 50

Not on the map. Why?

Abstract (arXiv)

We consider the privacy-preserving machine learning (ML) setting where the trained model must satisfy differential privacy (DP) with respect to the labels of the training examples. We propose two novel approaches based on, respectively, the Laplace mechanism and the PATE framework, and demonstrate their effectiveness on standard benchmarks. While recent work by Ghazi et al. proposed Label DP schemes based on a randomized response mechanism, we argue that additive Laplace noise coupled with Bayesian inference (ALIBI) is a better fit for typical ML tasks. Moreover, we show how to achieve very strong privacy levels in some regimes, with our adaptation of the PATE framework that builds on recent advances in semi-supervised learning. We complement theoretical analysis of our algorithms' privacy guarantees with empirical evaluation of their memorization properties. Our evaluation suggests that comparing different algorithms according to their provable DP guarantees can be misleading and favor a less private algorithm with a tighter analysis. Code for implementation of algorithms and memorization attacks is available from https://github.com/facebookresearch/label_dp_antipodes.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 23 of its 38 references that are in the map. First posted on arXiv 7 Jun 2021.

Builds on

23 of 38 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Wide Residual Networkscited by 3,057
  3. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  7. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  8. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  9. 2017A Closer Look at Memorization in Deep Networkscited by 936
  10. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  11. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  12. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
11 more
  1. 2018Scalable Private Learning with PATEcited by 266
  2. 2014RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Responsecited by 262
  3. 2014Learning with Pseudo-Ensemblescited by 241
  4. 2020Auditing Differentially Private Machine Learning: How Private is Private SGD?cited by 213
  5. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  6. 2021Deep Learning with Label Differential Privacycited by 113
  7. 2014Preserving Statistical Validity in Adaptive Data Analysiscited by 102
  8. 2018Private Selection from Private Candidatescited by 75
  9. 2013Projection onto the probability simplex: An efficient algorithm with a simple proof, and an applicationcited by 65
  10. 2014Private Learning and Sanitization: Pure vs. Approximate Differential Privacycited by 62
  11. 2019That which we call privatecited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 15; 2023: 12; 2024: 10; 2025: 6; 2026: 6.

50 papers.

Sort:

By area:

  1. 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 237
  2. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  3. 2023Privacy Auditing with One (1) Training Runcited by 119
  4. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  5. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 94
  6. 2022Synthetic Data -- what, why and how?cited by 66
  7. 2022Differential Privacy and Fairness in Decisions and Learning Tasks: A Surveycited by 47
  8. 2022SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learningcited by 47
  9. 2022Bayesian Estimation of Differential Privacycited by 31
  10. 2022Differentially Private Label Protection in Split Learningcited by 25
  11. 2023Unleashing the Power of Randomization in Auditing Differentially Private MLcited by 19
  12. 2022Regression with Label Differential Privacycited by 14
38 more
  1. 2024Preference Poisoning Attacks on Reward Model Learningcited by 14
  2. 2023DPMLBench: Holistic Evaluation of Differentially Private Machine Learningcited by 13
  3. 2023Differentially Private Reward Estimation with Preference Feedbackcited by 12
  4. 2022Does Label Differential Privacy Prevent Label Inference Attacks?cited by 10
  5. 2022Improving Privacy-Preserving Vertical Federated Learning by Efficient Communication with ADMMcited by 10
  6. 2022GeoPointGAN: Synthetic Spatial Data with Local Label Differential Privacycited by 9
  7. 2023Learning From Biased Soft Labelscited by 8
  8. 2025How Well Can Differential Privacy Be Audited in One Run?cited by 8
  9. 2022Private Ad Modeling with DP-SGDcited by 6
  10. 2023A Note On Interpreting Canary Exposurecited by 6
  11. 2023Optimal Unbiased Randomizers for Regression with Label Differential Privacycited by 6
  12. 2024Training Differentially Private Ad Prediction Models with Semi-Sensitive Featurescited by 6
  13. 2024Certification for Differentially Private Prediction in Gradient-Based Trainingcited by 6
  14. 2022FEL: High Capacity Learning for Recommendation and Ranking via Federated Ensemble Learningcited by 4
  15. 2022Privacy in Practice: Private COVID-19 Detection in X-Ray Images (Extended Version)cited by 4
  16. 2023Quantifying Overfitting: Evaluating Neural Network Performance through Analysis of Null Spacecited by 4
  17. 2024Retraining with Predicted Hard Labels Provably Increases Model Accuracycited by 4
  18. 2025The Hitchhiker's Guide to Efficient, End-to-End, and Tight DP Auditingcited by 4
  19. 2024Locally Private Estimation with Public Featurescited by 3
  20. 2022In Differential Privacy, There is Truth: On Vote Leakage in Ensemble Private Learningcited by 2
  21. 2024Enhancing Learning with Label Differential Privacy by Vector Approximationcited by 2
  22. 2024Auditing Privacy Mechanisms via Label Inference Attackscited by 2
  23. 2025Machine Learning with Privacy for Protected Attributescited by 2
  24. 2023Label Differential Privacy via Aggregationcited by 1
  25. 2023Does Differential Privacy Prevent Backdoor Attacks in Practice?cited by 1
  26. 2024Assessing the Impact of Image Dataset Features on Privacy-Preserving Machine Learningcited by 1
  27. 2026RPWithPrior: Label Differential Privacy in Regressioncited by 1
  28. 2023Private Gradient Estimation is Useful for Generative Modelingcited by 0
  29. 2024TablePuppet: A Generic Framework for Relational Federated Learningcited by 0
  30. 2024Privacy-Preserving Student Learning with Differentially Private Data-Free Distillationcited by 0
  31. 2025On Theoretical Limits of Learning with Label Differential Privacycited by 0
  32. 2025BLIA: Detect model memorization in binary classification model through passive Label Inference attackcited by 0
  33. 2025Observational Auditing of Label Privacycited by 0
  34. 2026Privacy-Preserving Model Transcription with Differentially Private Synthetic Distillationcited by 0
  35. 2026BlockRR: A Unified Framework of RR-type Algorithms for Label Differential Privacycited by 0
  36. 2026Private Learning with Public Feature Conditioningcited by 0
  37. 2026End-to-End Differential Privacy in Training Deep Neural Network Classifierscited by 0
  38. 2026Private Direct Preference Optimization for LLM Alignmentcited by 0