mlsanityData to 7 Oct 2026

SA-ADP: Sensitivity-Aware Adaptive Differential Privacy for Large Language Models

arXiv 2512.01748 ↗First posted 1 Dec 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Despite advances in the use of large language models (LLMs) in downstream tasks, their ability to memorize information has raised privacy concerns. Therefore, protecting personally identifiable information (PII) during LLM training remains a fundamental challenge. Conventional methods like Differential Privacy-Stochastic Gradient Descent (DP-SGD) provide robust privacy protection via uniform noising, protecting PII regardless of its distinct sensitivity. This comes at the expense of the model's utility, leading to a trade-off. In this paper, we propose SA-ADP, a sensitivity-aware approach that allocates noise based on the sensitivity of individual PII. We evaluated our method on four datasets (ABCD, CUSTOMERSIM, Wikitext-2, and UNSW-NB15 ). Our results show that SA-ADP achieves results comparable to the baseline (No-DP) and the conventional DP-SGD. This means that our method did not degrade the model's utility while still maintaining strong privacy protection.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 23 references that are in the map. First posted on arXiv 1 Dec 2025.

Builds on

13 of 23 references.

References · 13

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2016Pointer Sentinel Mixture Modelscited by 2,992
  3. 2016Deep Learning with Differential Privacycited by 2,682
  4. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  5. 2023Beyond Memorization: Violating Privacy Via Inference with Large Language Modelscited by 167
  6. 2021Selective Differential Privacy for Language Modelingcited by 61
  7. 2022Just Fine-tune Twice: Selective Differential Privacy for Large Language Modelscited by 32
  8. 2024Fine-Tuning Large Language Models with User-Level Differential Privacycited by 28
  9. 2023What Are the Chances? Explaining the Epsilon Parameter in Differential Privacycited by 13
  10. 2025SoK: Machine Unlearning for Large Language Modelscited by 9
  11. 2025Revisiting Privacy, Utility, and Efficiency Trade-offs when Fine-Tuning Large Language Modelscited by 5
  12. 2021DiPSeN: Differentially Private Self-normalizing Neural Networks For Adversarial Robustness in Federated Learningcited by 2
1 more
  1. 2024Training Data for Large Language Modelcited by 1

Built on by

Not cited yet.