mlsanityData to 13 Oct 2026

Differential Privacy for Text Analytics via Natural Text Sanitization

arXiv 2106.01221 ↗First posted 2 Jun 2021cs.CL cs.CRcited by 76

Not on the map. Why?

Abstract (arXiv)

Texts convey sophisticated knowledge. However, texts also convey sensitive information. Despite the success of general-purpose language models and domain-specific mechanisms with differential privacy (DP), existing text sanitization mechanisms still provide low utility, as cursed by the high-dimensional text representation. The companion issue of utilizing sanitized texts for downstream analytics is also under-explored. This paper takes a direct approach to text sanitization. Our insight is to consider both sensitivity and similarity via our new local DP notion. The sanitized texts also contribute to our sanitization-aware pretraining and fine-tuning, enabling privacy-preserving natural language processing over the BERT language model with promising utility. Surprisingly, the high utility does not boost up the success rate of inference attacks.

Key facts. Cited by 76 papers in the map (data through 13 Oct 2026). Builds on 23 of its 48 references that are in the map. First posted on arXiv 2 Jun 2021.

Builds on

23 of 48 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,880
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,273
  4. 2020Extracting Training Data from Large Language Modelscited by 2,181
  5. 2019Publicly Available Clinical BERT Embeddingscited by 999
  6. 2020Information Leakage in Embedding Modelscited by 203
  7. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 168
  8. 2017Controllable Invariance through Adversarial Feature Learningcited by 127
  9. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 118
  10. 2018Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardwarecited by 102
  11. 2018Towards Robust and Privacy-preserving Text Representationscited by 75
  12. 2020FALCON: Honest-Majority Maliciously Secure Framework for Private Deep Learningcited by 74
11 more
  1. 2020Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairnesscited by 67
  2. 2018Privacy-preserving Neural Representations of Textcited by 54
  3. 2019Leveraging Hierarchical Representations for Preserving Privacy and Utility in Textcited by 50
  4. 2018MedSTS: A Resource for Clinical Semantic Textual Similaritycited by 39
  5. 2020TextHide: Tackling Data Privacy in Language Understanding Taskscited by 33
  6. 2020Towards Differentially Private Text Representationscited by 29
  7. 2019Glyph: Fast and Accurately Training Deep Neural Networks on Encrypted Datacited by 28
  8. 2020Is Private Learning Possible with Instance Encoding?cited by 22
  9. 2018SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Miningcited by 21
  10. 2018Utility-Optimized Local Differential Privacy Mechanisms for Distribution Estimationcited by 12
  11. 2017Marginal Release Under Local Differential Privacycited by 9

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 7; 2024: 25; 2025: 25; 2026: 13.

76 papers.

Sort:

By area:

  1. 2024Machine Unlearning of Pre-trained Large Language Modelscited by 119
  2. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  3. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  4. 2023DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Passcited by 66
  5. 2021Selective Differential Privacy for Language Modelingcited by 61
  6. 2022A Customized Text Sanitization Mechanism with Differential Privacycited by 51
  7. 2024LLM-PBE: Assessing Data Privacy in Large Language Modelscited by 41
  8. 2024Privacy Preserving Prompt Engineering: A Surveycited by 25
  9. 2024DP-MLM: Differentially Private Text Rewriting Using Masked Language Modelscited by 24
  10. 2023Differentially Private Natural Language Models: Recent Advances and Future Directionscited by 20
  11. 2023Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymizationcited by 18
  12. 2022Sentence-level Privacy for Document Embeddingscited by 16
38 more
  1. 2024Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directionscited by 14
  2. 2024A Comparative Analysis of Word-Level Metric Differential Privacy: Benchmarking The Privacy-Utility Trade-offcited by 13
  3. 2024No Free Lunch Theorem for Privacy-Preserving LLM Inferencecited by 13
  4. 2024The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Promptscited by 13
  5. 2024Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domainscited by 12
  6. 2024Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generationcited by 11
  7. 20241-Diffractor: Efficient and Utility-Preserving Text Obfuscation Leveraging Word-Level Metric Differential Privacycited by 10
  8. 2024EmojiPrompt: Generative Prompt Obfuscation for Privacy-Preserving Communication with Cloud-based LLMscited by 9
  9. 2024Just Rewrite It Again: A Post-Processing Method for Enhanced Semantic Similarity and Privacy Preservation of Differentially Private Rewritten Textcited by 9
  10. 2024LLM-based Privacy Data Augmentation Guided by Knowledge Distillation with a Distribution Tutor for Medical Text Classificationcited by 8
  11. 2024Casper: Prompt Sanitization for Protecting User Privacy in Web-Based Large Language Modelscited by 8
  12. 2025On the Impact of Noise in Differentially Private Text Rewritingcited by 8
  13. 2024Privacy-Preserving Language Model Inference with Instance Obfuscationcited by 7
  14. 2025Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)cited by 6
  15. 2022Cross-Network Social User Embedding with Hybrid Differential Privacy Guaranteescited by 5
  16. 2023CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without Full Large Language Modelcited by 5
  17. 2024Locally Differentially Private In-Context Learningcited by 5
  18. 2025Spend Your Budget Wisely: Towards an Intelligent Distribution of the Privacy Budget in Differentially Private Text Rewritingcited by 5
  19. 2025Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacycited by 5
  20. 2025DP-Fusion: Token-Level Differentially Private Inference for Large Language Modelscited by 5
  21. 2025VFLAIR-LLM: A Comprehensive Framework and Benchmark for Split Learning of LLMscited by 5
  22. 2025How do we measure privacy in text? A survey of text anonymization metricscited by 5
  23. 2023To share or not to share: What risks would laypeople accept to give sensitive data to differentially-private NLP systems?cited by 3
  24. 2024Private Language Models via Truncated Laplacian Mechanismcited by 3
  25. 2024ExpShield: Safeguarding Web Text from Unauthorized Crawling and LLM Exploitationcited by 3
  26. 2025MaskSQL: Safeguarding Privacy for LLM-Based Text-to-SQL via Abstractioncited by 3
  27. 2023PrIeD-KIE: Towards Privacy Preserved Document Key Information Extractioncited by 2
  28. 2024A Collocation-based Method for Addressing Challenges in Word-level Metric Differential Privacycited by 2
  29. 2025Improving Privacy Benefits of Redactioncited by 2
  30. 2025DP-GTR: Differentially Private Prompt Protection via Group Text Rewritingcited by 2
  31. 2025Inspecting the Representation Manifold of Differentially-Private Textcited by 2
  32. 2025Anti-adversarial Learning: Desensitizing Prompts for Large Language Modelscited by 2
  33. 2026Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)cited by 2
  34. 2026From Prompts to Responses: Dual-Sided Data Leakage and Defense in Split Large Language Modelscited by 2
  35. 2025Investigating User Perspectives on Differentially Private Text Privatizationcited by 1
  36. 2025Empirical Calibration and Metric Differential Privacy in Language Modelscited by 1
  37. 2025The Double-edged Sword of LLM-based Data Reconstruction: Understanding and Mitigating Contextual Vulnerability in Word-level Differential Privacy Text Sanitizationcited by 1
  38. 2025Towards Confidential and Efficient LLM Inference with Dual Privacy Protectioncited by 1

50 most cited of 76 All 76 (JSON)