mlsanityData to 7 Oct 2026

Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions

arXiv 2412.16504 ↗First posted 21 Dec 2024cs.AIcited by 14

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Fine-tuning has emerged as a critical process in leveraging Large Language Models (LLMs) for specific downstream tasks, enabling these models to achieve state-of-the-art performance across various domains. However, the fine-tuning process often involves sensitive datasets, introducing privacy risks that exploit the unique characteristics of this stage. In this paper, we provide a comprehensive survey of privacy challenges associated with fine-tuning LLMs, highlighting vulnerabilities to various privacy attacks, including membership inference, data extraction, and backdoor attacks. We further review defense mechanisms designed to mitigate privacy risks in the fine-tuning phase, such as differential privacy, federated learning, and knowledge unlearning, discussing their effectiveness and limitations in addressing privacy risks and maintaining model utility. By identifying key gaps in existing research, we highlight challenges and propose directions to advance the development of privacy-preserving methods for fine-tuning LLMs, promoting their responsible use in diverse applications.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 35 of its 48 references that are in the map. First posted on arXiv 21 Dec 2024.

Builds on

35 of 48 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps14 Mar
  2. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration10 Nov
  3. 2023Analyzing Leakage of Personally Identifiable Information in Language Models1 Feb
  4. 2021Membership Inference Attack Susceptibility of Clinical Language Models16 Apr

Other references · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  4. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  7. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
  8. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
  9. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  10. 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 366
  11. 2023Poisoning Language Models During Instruction Tuningcited by 254
  12. 2023Unlearn What You Want to Forget: Efficient Unlearning for LLMscited by 211
19 more
  1. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 184
  2. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 143
  3. 2024On Protecting the Data Privacy of Large Language Models (LLMs): A Surveycited by 91
  4. 2019Analyzing Information Leakage of Updates to Natural Language Modelscited by 89
  5. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 86
  6. 2023Offsite-Tuning: Transfer Learning without Full Modelcited by 77
  7. 2021Differential Privacy for Text Analytics via Natural Text Sanitizationcited by 74
  8. 2022Recovering Private Text in Federated Learning of Language Modelscited by 67
  9. 2022Differentially Private Optimization on Large Model at Small Costcited by 65
  10. 2019Leveraging Hierarchical Representations for Preserving Privacy and Utility in Textcited by 49
  11. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 48
  12. 2023Privacy-Preserving Parameter-Efficient Fine-Tuning for Large Language Model Servicescited by 43
  13. 2023Controlling the Extraction of Memorized Data from Large Language Models via Prompt-Tuningcited by 40
  14. 2022Privately Fine-Tuning Large Language Models with Differential Privacycited by 39
  15. 2023Hide and Seek (HaS): A Lightweight Framework for Prompt Privacy Protectioncited by 37
  16. 2024Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Modelscited by 28
  17. 2023DP-BART for Privatized Text Rewriting under Local Differential Privacycited by 27
  18. 2024Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutionscited by 14
  19. 2024Safely Learning with Private Data: A Federated Learning Framework for Large Language Modelcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 4.

14 papers.

Sort:

By area:

  1. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  2. 2025Membership Inference Attacks on Large-Scale Models: A Surveycited by 17
  3. 2025The Future of Continual Learning in the Era of Foundation Models: Three Key Directionscited by 16
  4. 2025Position: Privacy Is Not Just Memorization!cited by 15
  5. 2025What Large Language Models Do Not Talk About: An Empirical Study of Moderation and Censorship Practicescited by 11
  6. 2025VeriLeaky: Navigating IP Protection vs Utility in Fine-Tuning for LLM-Driven Verilog Codingcited by 10
  7. 2025SALAD: Systematic Assessment of Machine Unlearning on LLM-Aided Hardware Designcited by 10
  8. 2025Jailbreak-Tuning: Models Efficiently Learn Jailbreak Susceptibilitycited by 6
  9. 2026Toward Enhancing Representation Learning in Federated Multi-Task Settingscited by 2
  10. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  11. 2026TuneShift-KD: Knowledge Distillation and Transfer for Fine-tuned Modelscited by 1
  12. 2025Federated Foundation Language Model Post-Training Should Focus on Open-Source Modelscited by 0
2 more
  1. 2026Stable and Privacy-Preserving Synthetic Educational Data with Empirical Marginals: A Copula-Based Approachcited by 0
  2. 2026FISGuard: Defending Against Membership Inference via Fixed Input Subspacescited by 0