mlsanityData to 7 Oct 2026

A Study of Privacy-preserving Language Modeling Approaches

arXiv 2508.15421 ↗First posted 21 Aug 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent developments in language modeling have increased their use in various applications and domains. Language models, often trained on sensitive data, can memorize and disclose this information during privacy attacks, raising concerns about protecting individuals' privacy rights. Preserving privacy in language models has become a crucial area of research, as privacy is one of the fundamental human rights. Despite its significance, understanding of how much privacy risk these language models possess and how it can be mitigated is still limited. This research addresses this by providing a comprehensive study of the privacy-preserving language modeling approaches. This study gives an in-depth overview of these approaches, highlights their strengths, and investigates their limitations. The outcomes of this study contribute to the ongoing research on privacy-preserving language modeling, providing valuable insights and outlining future research directions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 37 references that are in the map. First posted on arXiv 21 Aug 2025.

Builds on

24 of 37 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016Deep Learning with Differential Privacycited by 2,682
  5. 2020Extracting Training Data from Large Language Modelscited by 2,144
  6. 2020Adaptive Federated Optimizationcited by 1,036
  7. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  8. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
  9. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  10. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  11. 2020Amnesiac Machine Learningcited by 305
  12. 2023Large Language Model Unlearningcited by 288
12 more
  1. 2020Information Leakage in Embedding Modelscited by 199
  2. 2022The Privacy Onion Effect: Memorization is Relativecited by 128
  3. 2023ProPILE: Probing Privacy Leakage in Large Language Modelscited by 127
  4. 2021Large-Scale Differentially Private BERTcited by 117
  5. 2021FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Taskscited by 84
  6. 2021Differential Privacy for Text Analytics via Natural Text Sanitizationcited by 74
  7. 2021Selective Differential Privacy for Language Modelingcited by 61
  8. 2022A Customized Text Sanitization Mechanism with Differential Privacycited by 51
  9. 2020TextHide: Tackling Data Privacy in Language Understanding Taskscited by 32
  10. 2022Just Fine-tune Twice: Selective Differential Privacy for Large Language Modelscited by 32
  11. 2023Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymizationcited by 18
  12. 2022Bootstrapping Text Anonymization Models with Distant Supervisioncited by 8

Built on by

Not cited yet.