mlsanityData to 14 Oct 2026

SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks

arXiv 2506.10424 ↗First posted 12 Jun 2025cs.CR cs.AIcited by 10

On the map: LLM safety and security → Membership inference attacks

Abstract (arXiv)

Large language models (LLMs) have achieved remarkable success and are widely adopted for diverse applications. However, fine-tuning these models often involves private or sensitive information, raising critical privacy concerns. In this work, we conduct the first comprehensive study evaluating the vulnerability of fine-tuned LLMs to membership inference attacks (MIAs). Our empirical analysis demonstrates that MIAs exploit the loss reduction during fine-tuning, making them highly effective in revealing membership information. These findings motivate the development of our defense. We propose SOFT (Selective data Obfuscation in LLM Fine-Tuning), a novel defense technique that mitigates privacy leakage by leveraging influential data selection with an adjustable parameter to balance utility preservation and privacy protection. Our extensive experiments span six diverse domains and multiple LLM architectures and scales. Results show that SOFT effectively reduces privacy risks while maintaining competitive model performance, offering a practical and scalable solution to safeguard sensitive information in fine-tuned LLMs.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 73 of its 99 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

73 of 99 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Data29 Sep
  2. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)25 Jun
  3. 2024Blind Baselines Beat Membership Inference Attacks for Foundation Models23 Jun
  4. 2024ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods23 Jun
  5. 2024LLM Dataset Inference: Did you train on my dataset?10 Jun
  6. 2024Do Membership Inference Attacks Work on Large Language Models?12 Feb
  7. 2023Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models23 Oct
  8. 2022Membership Inference Attacks by Exploiting Loss Trajectory31 Aug
  9. 2021Membership Inference Attacks From First Principles7 Dec
  10. 2021Membership Inference Attack Susceptibility of Clinical Language Models16 Apr

Other references · 63

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  11. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  12. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
38 more
  1. 2016Deep Learning with Differential Privacycited by 2,728
  2. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  4. 2020Extracting Training Data from Large Language Modelscited by 2,189
  5. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  7. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  8. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  9. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  10. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
  11. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  12. 2023Scaling Data-Constrained Language Modelscited by 403
  13. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  14. 2020Label-Only Membership Inference Attackscited by 388
  15. 2023Detecting Pretraining Data from Large Language Modelscited by 387
  16. 2024LoRA Learns Less and Forgets Lesscited by 328
  17. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 324
  18. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  19. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 230
  20. 2021Machine Unlearning of Features and Labelscited by 229
  21. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  22. 2024Data Engineering for Scaling Language Models to 128K Contextcited by 186
  23. 2024How to Train Long-Context Language Models (Effectively)cited by 146
  24. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 142
  25. 2024HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuningcited by 123
  26. 2021Large-Scale Differentially Private BERTcited by 118
  27. 2023Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learningcited by 115
  28. 2024LoRA vs Full Fine-tuning: An Illusion of Equivalencecited by 109
  29. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 103
  30. 2023KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignmentcited by 99
  31. 2022The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymizationcited by 95
  32. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 83
  33. 2024DE-COP: Detecting Copyrighted Content in Language Models Training Datacited by 74
  34. 2023Bag of Tricks for Training Data Extraction from Language Modelscited by 70
  35. 2024LoFiT: Localized Fine-tuning on LLM Representationscited by 57
  36. 2024LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Reportcited by 47
  37. 2019One Epoch Is All You Needcited by 41
  38. 2023The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Riskscited by 36

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.

  1. 2026Window-based Membership Inference Attacks Against Fine-tuned Large Language Modelscited by 7
  2. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  3. 2026Membership Inference Attacks Against Fine-tuned Diffusion Language Modelscited by 5
  4. 2025A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challengescited by 3
  5. 2025Membership Inference Attacks on Tokenizers of Large Language Modelscited by 3
  6. 2025AnalogFed: Privacy-Preserving Discovery of Analog Circuits at Scale with Federated Generative AIcited by 0
  7. 2026What Hard Tokens Reveal: Exploiting Low-confidence Tokens for Membership Inference Attacks against Large Language Modelscited by 0
  8. 2026PrivMedChat: End-to-End Differentially Private RLHF for Medical Dialogue Systemscited by 0
  9. 2026Privacy-Preserving Machine Learning for IoT: A Cross-Paradigm Survey and Future Roadmapcited by 0
  10. 2026When T2I Synthetic Data Backfires: Amplified Privacy Risks in Real-Synthetic Mix Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directions
  2. 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions
  3. 2024Order of Magnitude Speedups for LLM Membership Inference
  4. 2025A Survey on Privacy Risks and Protection in Large Language Models
  5. 2026On the Privacy of LLMs: An Ablation Study
  6. 2025Ensemble Privacy Defense for Knowledge-Intensive LLMs against Membership Inference Attacks
  7. 2024SoK: Reducing the Vulnerability of Fine-tuned Language Models to Membership Inference Attacks
  8. 2025Membership Inference Attacks on Large-Scale Models: A Survey