mlsanityData to 7 Oct 2026

Scalable Private Learning with PATE

arXiv 1802.08908 ↗First posted 24 Feb 2018stat.ML cs.CR cs.LGcited by 266

Not on the map. Why?

Cited by 21 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The rapid adoption of machine learning has increased concerns about the privacy implications of machine learning models trained on sensitive data, such as medical records or other personal information. To address those concerns, one promising approach is Private Aggregation of Teacher Ensembles, or PATE, which transfers to a "student" model the knowledge of an ensemble of "teacher" models, with intuitive privacy provided by training teachers on disjoint data and strong privacy guaranteed by noisy aggregation of teachers' answers. However, PATE has so far been evaluated only on simple classification tasks like MNIST, leaving unclear its utility when applied to larger-scale learning tasks and real-world datasets. In this work, we show how PATE can scale to learning tasks with large numbers of output classes and uncurated, imbalanced training data with errors. For this, we introduce new noisy aggregation mechanisms for teacher ensembles that are more selective and add less noise, and prove their tighter differential-privacy guarantees. Our new mechanisms build on two insights: the chance of teacher consensus is increased by using more concentrated noise and, lacking consensus, no answer need be given to a student. The consensus answers used are more likely to be correct, offer better intuitive privacy, and incur lower-differential privacy cost. Our evaluation shows our mechanisms improve on the original PATE on all measures, and scale to larger tasks with both high utility and very strong privacy (varepsilon < 1.0).

Key facts. Cited by 266 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 48; 2022: 65; 2023: 61; 2024: 44; 2025: 23; 2026: 25.

266 papers.

Sort:

By area:

  1. 2021Membership Inference Attacks From First Principlescited by 839
  2. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  3. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  4. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  5. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  6. 2021Adversary Instantiation: Lower Bounds for Differentially Private Machine Learningcited by 193
  7. 2021Adaptive Machine Unlearningcited by 164
  8. 2022Reconstructing Training Data with Informed Adversariescited by 143
  9. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  10. 2022Differentially Private Diffusion Modelscited by 118
  11. 2021Deep Learning with Label Differential Privacycited by 113
  12. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 94
38 more
  1. 2021Large Scale Private Learning via Low-rank Reparametrizationcited by 87
  2. 2021Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learningcited by 86
  3. 2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 81
  4. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 80
  5. 2021FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learningcited by 67
  6. 2022Synthetic Data -- what, why and how?cited by 66
  7. 2021Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecturecited by 63
  8. 2021Selective Differential Privacy for Language Modelingcited by 61
  9. 2021Privacy-Preserving Machine Learning: Methods, Challenges and Directionscited by 58
  10. 2022Mixed Differential Privacy in Computer Visioncited by 53
  11. 2021Membership Inference Attacks Against Recommender Systemscited by 52
  12. 2021Membership Inference Attack Susceptibility of Clinical Language Modelscited by 50
  13. 2021Antipodes of Label Differential Privacy: PATE and ALIBIcited by 50
  14. 2022Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregationcited by 50
  15. 2021Public Data-Assisted Mirror Descent for Private Model Trainingcited by 49
  16. 2021Robin Hood and Matthew Effects: Differential Privacy Has Disparate Impact on Synthetic Datacited by 48
  17. 2022Differential Privacy and Fairness in Decisions and Learning Tasks: A Surveycited by 47
  18. 2021Survey: Leakage and Privacy at Inference Timecited by 44
  19. 2021Gradient-Leakage Resilient Federated Learningcited by 40
  20. 2022Large Scale Transfer Learning for Differentially Private Image Classificationcited by 40
  21. 2021Leveraging Public Data for Practical Private Query Releasecited by 39
  22. 2024Privacy-Preserving Retrieval-Augmented Generation with Differential Privacycited by 38
  23. 2023SoK: Privacy-Preserving Data Synthesiscited by 37
  24. 2021Quantifying and Mitigating Privacy Risks of Contrastive Learningcited by 35
  25. 2022RelaxLoss: Defending Membership Inference Attacks without Losing Utilitycited by 35
  26. 2022Private Set Generation with Discriminative Informationcited by 34
  27. 2021CaPC Learning: Confidential and Private Collaborative Learningcited by 33
  28. 2021Releasing Graph Neural Networks with Differential Privacy Guaranteescited by 32
  29. 2022Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacycited by 32
  30. 2022Just Fine-tune Twice: Selective Differential Privacy for Large Language Modelscited by 32
  31. 2022On the Privacy Risks of Algorithmic Recoursecited by 32
  32. 2021DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregationcited by 31
  33. 2022Submix: Practical Private Prediction for Large-Scale Language Modelscited by 31
  34. 2022Private Estimation with Public Datacited by 31
  35. 2021Practical Defences Against Model Inversion Attacks for Split Neural Networkscited by 29
  36. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  37. 2023Integration of Large Language Models and Federated Learningcited by 28
  38. 2025A Comprehensive Survey of Synthetic Tabular Data Generationcited by 27

50 most cited of 266 All 266 (JSON)