mlsanityData to 7 Oct 2026

Large Language Models Can Be Strong Differentially Private Learners

arXiv 2110.05679 ↗First posted 12 Oct 2021cs.LG cs.CLcited by 405

Not on the map. Why?

Cited by 37 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Differentially Private (DP) learning has seen limited success for building large deep learning models of text, and straightforward attempts at applying Differentially Private Stochastic Gradient Descent (DP-SGD) to NLP tasks have resulted in large performance drops and high computational overhead. We show that this performance drop can be mitigated with (1) the use of large pretrained language models; (2) non-standard hyperparameters that suit DP optimization; and (3) fine-tuning objectives which are aligned with the pretraining procedure. With the above, we obtain NLP models that outperform state-of-the-art DP-trained models under the same privacy budget and strong non-private baselines -- by directly fine-tuning pretrained models with DP optimization on moderately-sized corpora. To address the computational challenge of running DP-SGD with large Transformers, we propose a memory saving technique that allows clipping in DP-SGD to run without instantiating per-example gradients for any linear layer in the model. The technique enables privately training Transformers with almost the same memory cost as non-private training at a modest run-time overhead. Contrary to conventional wisdom that DP optimization fails at learning high-dimensional models (due to noise that scales with dimension) empirical results reveal that private learning with pretrained language models doesn't tend to suffer from dimension-dependent performance degradation. Code to reproduce results can be found at https://github.com/lxuechen/private-transformers.

Key facts. Cited by 405 papers in the map (data through 7 Oct 2026). Builds on 70 of its 107 references that are in the map. First posted on arXiv 12 Oct 2021.

Builds on

70 of 107 references.

References · 70

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  10. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  11. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
38 more
  1. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  4. 2020Extracting Training Data from Large Language Modelscited by 2,144
  5. 2019Language Models as Knowledge Bases?cited by 1,922
  6. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  8. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  9. 2017Mixed Precision Trainingcited by 1,135
  10. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  11. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  12. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  13. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  14. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  15. 2021GPT Understands, Toocited by 715
  16. 2017Renyi Differential Privacycited by 643
  17. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
  18. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
  19. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  20. 2018Scalable Private Learning with PATEcited by 266
  21. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 252
  22. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  23. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
  24. 2020AdapterDrop: On the Efficiency of Adapters in Transformerscited by 223
  25. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  26. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 178
  27. 2021Numerical Composition of Differential Privacycited by 174
  28. 2018Neural Approaches to Conversational AIcited by 164
  29. 2020DART: Open-Domain Structured Data Record to Text Generationcited by 144
  30. 2020DP-CGAN: Differentially Private Synthetic Data and Label Generationcited by 118
  31. 2020GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generatorscited by 117
  32. 2021Large-Scale Differentially Private BERTcited by 117
  33. 2019Deep Learning with Gaussian Differential Privacycited by 112
  34. 2014Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Boundscited by 104
  35. 2021Hyperparameter Tuning with Renyi Differential Privacycited by 104
  36. 2020InstaHide: Instance-hiding Schemes for Private Distributed Learningcited by 90
  37. 2020Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identificationcited by 88
  38. 2021Large Scale Private Learning via Low-rank Reparametrizationcited by 87

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 60; 2023: 99; 2024: 102; 2025: 83; 2026: 56.

405 papers.

Sort:

By area:

  1. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
  2. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  3. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  4. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  5. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  6. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  7. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 218
  8. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  9. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  10. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 211
  11. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 195
  12. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
38 more
  1. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  2. 2024Improving LoRA in Privacy-preserving Federated Learningcited by 158
  3. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  4. 2022Differentially Private Diffusion Modelscited by 118
  5. 2024Machine Unlearning of Pre-trained Large Language Modelscited by 118
  6. 2023Foundation Models and Fair Usecited by 112
  7. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  8. 2023On Provable Copyright Protection for Generative Modelscited by 97
  9. 2022The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymizationcited by 94
  10. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 94
  11. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibrationcited by 90
  12. 2022Toward Training at ImageNet Scale with Differential Privacycited by 87
  13. 2022Automatic Clipping: Differentially Private Deep Learning Made Easier and Strongercited by 87
  14. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 86
  15. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  16. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 80
  17. 2024Data Shapley in One Training Runcited by 76
  18. 2023DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Modelscited by 75
  19. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Textcited by 74
  20. 2024Safeguarding Large Language Models: A Surveycited by 73
  21. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  22. 2022Recovering Private Text in Federated Learning of Language Modelscited by 67
  23. 2022Differentially Private Optimization on Large Model at Small Costcited by 65
  24. 2023Harnessing large-language models to generate private synthetic textcited by 65
  25. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 64
  26. 2022The Limits of Word Level Differential Privacycited by 63
  27. 2023Differentially Private Synthetic Data via Foundation Model APIs 1: Imagescited by 63
  28. 2023Locally Differentially Private Document Generation Using Zero Shot Promptingcited by 63
  29. 2024Building Guardrails for Large Language Modelscited by 61
  30. 2023"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agentscited by 60
  31. 2022Mixed Differential Privacy in Computer Visioncited by 53
  32. 2022Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Modelscited by 52
  33. 2022Exploring the Limits of Differentially Private Deep Learning with Group-wise Clippingcited by 52
  34. 2022A Customized Text Sanitization Mechanism with Differential Privacycited by 51
  35. 2022TAN Without a Burn: Scaling Laws of DP-SGDcited by 51
  36. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Modelscited by 51
  37. 2023Privacy-Preserving In-Context Learning for Large Language Modelscited by 50
  38. 2021Public Data-Assisted Mirror Descent for Private Model Trainingcited by 49

50 most cited of 405 All 405 (JSON)