mlsanityData to 7 Oct 2026

Provably Protecting Fine-Tuned LLMs from Training Data Extraction while Preserving Utility

arXiv 2602.00688 ↗First posted 31 Jan 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Fine-tuning large language models (LLMs) on sensitive datasets raises privacy concerns, as training data extraction (TDE) attacks can expose highly confidential information. Existing defenses against such attacks either lack formal privacy guarantees or incur substantial utility degradation. We observe that fine-tuning induces widespread probability shifts, yet preserving only a small subset of influential token-level deviations is sufficient; the remaining shifts can be aggressively smoothed with minimal impact on utility. Motivated by this insight, we propose SCP-Deltar, a Near Access Freeness (NAF)-based algorithm that operates on relative probabilities and explicitly smooths low-impact tokens using a base model. SCP-Deltar achieves orders-of-magnitude better theoretical bounds than existing NAF based methods and provides strong empirical protection against TDE attacks with minimal performance loss.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 40 references that are in the map. First posted on arXiv 31 Jan 2026.

Builds on

27 of 40 references.

References · 27

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2018Hierarchical Neural Story Generationcited by 1,182
  5. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  6. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  7. 2017Selective Classification for Deep Neural Networkscited by 799
  8. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  9. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  10. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  11. 2019White-box vs Black-box: Bayes Optimal Strategies for Membership Inferencecited by 265
  12. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
15 more
  1. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  2. 2023On Provable Copyright Protection for Generative Modelscited by 97
  3. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibrationcited by 90
  4. 2024Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMscited by 50
  5. 2022A Critical Review on the Use (and Misuse) of Differential Privacy in Machine Learningcited by 27
  6. 2024Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Textscited by 26
  7. 2024Purifying Large Language Models by Ensembling a Small Language Modelcited by 25
  8. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Datacited by 21
  9. 2025LoRA-Leak: Membership Inference Attacks Against LoRA Fine-tuned Language Modelscited by 7
  10. 2024Copyright-Protected Language Generation via Adaptive Model Fusioncited by 6
  11. 2025Can Differentially Private Fine-tuning LLMs Protect Against Privacy Attacks?cited by 4
  12. 2024DOMBA: Double Model Balancing for Access-Controlled Language Models via Minimum-Bounded Aggregationcited by 2
  13. 2025Consensus Sampling for Safer Generative AIcited by 2
  14. 2025Leaking LoRa: An Evaluation of Password Leaks and Knowledge Storage in Large Language Modelscited by 1
  15. 2025Blameless Users in a Clean Room: Defining Copyright Protection for Generative Modelscited by 1

Built on by

Not cited yet.