mlsanityData to 7 Oct 2026

When differential privacy meets NLP: The devil is in the detail

arXiv 2109.03175 ↗First posted 7 Sep 2021cs.CL cs.CRcited by 25

Not on the map. Why?

Abstract (arXiv)

Differential privacy provides a formal approach to privacy of individuals. Applications of differential privacy in various scenarios, such as protecting users' original utterances, must satisfy certain mathematical properties. Our contribution is a formal analysis of ADePT, a differentially private auto-encoder for text rewriting (Krishna et al, 2021). ADePT achieves promising results on downstream tasks while providing tight privacy guarantees. Our proof reveals that ADePT is not differentially private, thus rendering the experimental results unsubstantiated. We also quantify the impact of the error in its private mechanism, showing that the true sensitivity is higher by at least factor 6 in an optimistic case of a very small encoder's dimension and that the amount of utterances that are not privatized could easily reach 100% of the entire dataset. Our intention is neither to criticize the authors, nor the peer-reviewing process, but rather point out that if differential privacy applications in NLP rely on formal guarantees, these should be outlined in full and put under detailed scrutiny.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 6 of its 8 references that are in the map. First posted on arXiv 7 Sep 2021.

Builds on

6 of 8 references.

References · 6

  1. 2017Renyi Differential Privacycited by 643
  2. 2019SoK: Differential Privaciescited by 33
  3. 2021ADePT: Auto-encoder based Differentially Private Text Transformationcited by 31
  4. 2020A Comprehensive Survey on Local Differential Privacy Toward Data Statistics and Analysiscited by 26
  5. 2021Privacy-Preserving Graph Convolutional Networks for Text Classificationcited by 19
  6. 2018When the signal is in the noise: Exploiting Diffix's Sticky Noisecited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 7; 2024: 3; 2025: 6; 2026: 1.

25 papers.

Sort:

By area:

  1. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  2. 2022The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymizationcited by 94
  3. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  4. 2022DP-Rewrite: Towards Reproducibility and Transparency in Differentially Private Text Rewritingcited by 28
  5. 2023DP-BART for Privatized Text Rewriting under Local Differential Privacycited by 27
  6. 2023Differentially Private Natural Language Models: Recent Advances and Future Directionscited by 20
  7. 2022Differential Privacy in Natural Language Processing: The Story So Farcited by 12
  8. 2024A Comparative Analysis of Word-Level Metric Differential Privacy: Benchmarking The Privacy-Utility Trade-offcited by 11
  9. 2022How reparametrization trick broke differentially-private text representation learningcited by 10
  10. 2022Privacy-Preserving Models for Legal Natural Language Processingcited by 10
  11. 2025LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Datacited by 10
  12. 2022Fair NLP Models with Differentially Private Text Encoderscited by 9
13 more
  1. 2023DP-NMT: Scalable Differentially-Private Machine Translationcited by 7
  2. 2025Spend Your Budget Wisely: Towards an Intelligent Distribution of the Privacy Budget in Differentially Private Text Rewritingcited by 5
  3. 2023Trade-Offs Between Fairness and Privacy in Language Modelingcited by 4
  4. 2023A Neighbourhood-Aware Differential Privacy Mechanism for Static Word Embeddingscited by 4
  5. 2023Neural Text Sanitization with Privacy Risk Indicators: An Empirical Analysiscited by 4
  6. 2025Generating Synthetic Data with Formal Privacy Guarantees: State of the Art and the Road Aheadcited by 4
  7. 2025SynBench: A Benchmark for Differentially Private Text Generationcited by 4
  8. 2024Private Language Models via Truncated Laplacian Mechanismcited by 3
  9. 2025Leveraging Semantic Triples for Private Document Generation with Local Differential Privacy Guaranteescited by 3
  10. 2023Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Modelscited by 1
  11. 2025Investigating User Perspectives on Differentially Private Text Privatizationcited by 1
  12. 2024IDT: Dual-Task Adversarial Attacks for Privacy Protectioncited by 0
  13. 2026Beyond Theoretical Bounds: Empirical Privacy Loss Calibration for Text Rewriting Under Local Differential Privacycited by 0