mlsanityData to 14 Oct 2026

Differentially Private Fine-tuning of Language Models

arXiv 2110.06500 ↗First posted 13 Oct 2021cs.LG cs.CL cs.CRcited by 400

Not on the map. Why?

Cited by 49 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-scale pre-trained language models, which achieve the state-of-the-art privacy versus utility tradeoffs on many standard NLP tasks. We propose a meta-framework for this problem, inspired by the recent success of highly parameter-efficient methods for fine-tuning. Our experiments show that differentially private adaptations of these approaches outperform previous private algorithms in three important dimensions: utility, privacy, and the computational and memory cost of private training. On many commonly studied datasets, the utility of private models approaches that of non-private models. For example, on the MNLI dataset we achieve an accuracy of 87.8% using RoBERTa-Large and 83.5% using RoBERTa-Base with a privacy budget of ε = 6.7. In comparison, absent privacy constraints, RoBERTa-Large achieves an accuracy of 90.2%. Our findings are similar for natural language generation tasks. Privately fine-tuning with DART, GPT-2-Small, GPT-2-Medium, GPT-2-Large, and GPT-2-XL achieve BLEU scores of 38.5, 42.0, 43.1, and 43.8 respectively (privacy budget of ε = 6.8,δ= 1e-5) whereas the non-private baseline is 48.1. All our experiments suggest that larger models are better suited for private fine-tuning: while they are well known to achieve superior accuracy non-privately, we find that they also better maintain their accuracy when privacy is introduced.

Key facts. Cited by 400 papers in the map (data through 14 Oct 2026). Builds on 49 of its 91 references that are in the map. First posted on arXiv 13 Oct 2021.

Builds on

49 of 91 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  11. 2016Deep Learning with Differential Privacycited by 2,728
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
37 more
  1. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  2. 2020Extracting Training Data from Large Language Modelscited by 2,189
  3. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  4. 2017Mixed Precision Trainingcited by 1,156
  5. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  6. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  7. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  8. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 671
  9. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  10. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 460
  11. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  12. 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 370
  13. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
  14. 2018Scalable Private Learning with PATEcited by 269
  15. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 253
  16. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 235
  17. 2020AdapterDrop: On the Efficiency of Adapters in Transformerscited by 224
  18. 2018Scaling Neural Machine Translationcited by 204
  19. 2021Numerical Composition of Differential Privacycited by 180
  20. 2020DART: Open-Domain Structured Data Record to Text Generationcited by 145
  21. 2021Large-Scale Differentially Private BERTcited by 118
  22. 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 104
  23. 2020Bypassing the Ambient Dimension: Private SGD with Gradient Subspace Identificationcited by 90
  24. 2021Large Scale Private Learning via Low-rank Reparametrizationcited by 88
  25. 2021Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learningcited by 86
  26. 2013Fingerprinting Codes and the Price of Approximate Differential Privacycited by 82
  27. 2020Training Production Language Models without Memorizing User Datacited by 68
  28. 2014Private Learning and Sanitization: Pure vs. Approximate Differential Privacycited by 63
  29. 2020Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorizationcited by 58
  30. 2020Differentially Private Language Models Benefit from Public Pre-trainingcited by 47
  31. 2020Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFTcited by 43
  32. 2019Limits of Private Learning with Access to Public Datacited by 39
  33. 2021Leveraging Public Data for Practical Private Query Releasecited by 39
  34. 2021Benchmarking Differential Privacy and Federated Learning for BERT Modelscited by 28
  35. 2020Private Query Release Assisted by Public Datacited by 27
  36. 2020Learning from Mixtures of Private and Public Populationscited by 19
  37. 2019Privately Answering Classification Queries in the Agnostic PAC Modelcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 48; 2023: 92; 2024: 105; 2025: 81; 2026: 69.

400 papers.

Sort:

By area:

  1. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  2. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 437
  3. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  4. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
  5. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  6. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 258
  7. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  8. 2022Unlocking High-Accuracy Differentially Private Image Classification through Scalecited by 224
  9. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 215
  10. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
  11. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  12. 2024Improving LoRA in Privacy-preserving Federated Learningcited by 161
38 more
  1. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 153
  2. 2023AIM: Adapting Image Models for Efficient Video Action Recognitioncited by 148
  3. 2024FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptationscited by 146
  4. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 131
  5. 2023Machine Learning for Synthetic Data Generation: A Reviewcited by 120
  6. 2022Differentially Private Diffusion Modelscited by 119
  7. 2023Foundation Models and Fair Usecited by 116
  8. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  9. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 100
  10. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibrationcited by 98
  11. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 96
  12. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 88
  13. 2022Toward Training at ImageNet Scale with Differential Privacycited by 87
  14. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 83
  15. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 82
  16. 2023LLMaAA: Making Large Language Models as Active Annotatorscited by 79
  17. 2024Safeguarding Large Language Models: A Surveycited by 78
  18. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  19. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Textcited by 76
  20. 2022Recovering Private Text in Federated Learning of Language Modelscited by 70
  21. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  22. 2023DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Passcited by 67
  23. 2023Federated Learning of Gboard Language Models with Differential Privacycited by 66
  24. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  25. 2022Differentially Private Optimization on Large Model at Small Costcited by 65
  26. 2023Differentially Private Synthetic Data via Foundation Model APIs 1: Imagescited by 65
  27. 2023Harnessing large-language models to generate private synthetic textcited by 65
  28. 2024Building Guardrails for Large Language Modelscited by 65
  29. 2021Differential privacy and robust statistics in high dimensionscited by 64
  30. 2022The Limits of Word Level Differential Privacycited by 63
  31. 2023Locally Differentially Private Document Generation Using Zero Shot Promptingcited by 63
  32. 2023"It's a Fair Game", or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agentscited by 60
  33. 2022Mixed Differential Privacy in Computer Visioncited by 54
  34. 2022TAN Without a Burn: Scaling Laws of DP-SGDcited by 52
  35. 2022Exploring the Limits of Differentially Private Deep Learning with Group-wise Clippingcited by 52
  36. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
  37. 2023Privacy-Preserving In-Context Learning for Large Language Modelscited by 50
  38. 2024UNDIAL: Self-Distillation with Adjusted Logits for Robust Unlearning in Large Language Modelscited by 50

50 most cited of 400 All 400 (JSON)