mlsanityData to 7 Oct 2026

Locally Differentially Private Document Generation Using Zero Shot Prompting

arXiv 2310.16111 ↗First posted 24 Oct 2023cs.CL cs.CR cs.LGcited by 63

Not on the map. Why?

Abstract (arXiv)

Numerous studies have highlighted the privacy risks associated with pretrained large language models. In contrast, our research offers a unique perspective by demonstrating that pretrained large language models can effectively contribute to privacy preservation. We propose a locally differentially private mechanism called DP-Prompt, which leverages the power of pretrained large language models and zero-shot prompting to counter author de-anonymization attacks while minimizing the impact on downstream utility. When DP-Prompt is used with a powerful language model like ChatGPT (gpt-3.5), we observe a notable reduction in the success rate of de-anonymization attacks, showing that it surpasses existing approaches by a considerable margin despite its simpler design. For instance, in the case of the IMDB dataset, DP-Prompt (with ChatGPT) perfectly recovers the clean sentiment F1 score while achieving a 46% reduction in author identification F1 score against static attackers and a 26% reduction against adaptive attackers. We conduct extensive experiments across six open-source large language models, ranging up to 7 billion parameters, to analyze various effects of the privacy-utility tradeoff.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 37 of its 67 references that are in the map. First posted on arXiv 24 Oct 2023.

Builds on

37 of 67 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
25 more
  1. 2020Extracting Training Data from Large Language Modelscited by 2,144
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  3. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  4. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  5. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  6. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  7. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  8. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  9. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  10. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  11. 2014RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Responsecited by 262
  12. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 115
  13. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  14. 2021TAG: Gradient Attack on Transformer-based Language Modelscited by 63
  15. 2022The Limits of Word Level Differential Privacycited by 63
  16. 2022LAMP: Extracting Text from Gradients with Language Model Priorscited by 52
  17. 2022A Customized Text Sanitization Mechanism with Differential Privacycited by 51
  18. 2020Differentially Private Language Models Benefit from Public Pre-trainingcited by 46
  19. 2023On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Researchcited by 44
  20. 2021TEM: High Utility Metric Differential Privacy on Textcited by 43
  21. 2017A4NT: Author Attribute Anonymity by Adversarial Training of Neural Machine Translationcited by 41
  22. 2020A Differentially Private Text Perturbation Method Using a Regularized Mahalanobis Metriccited by 41
  23. 2019Misleading Authorship Attribution of Source Code using Adversarial Learningcited by 33
  24. 2022A Critical Review on the Use (and Misuse) of Differential Privacy in Machine Learningcited by 27
  25. 2021Differentially Private Imaging via Latent Space Manipulationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 20; 2025: 24; 2026: 17.

63 papers.

Sort:

By area:

  1. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)cited by 159
  2. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Textcited by 74
  3. 2023DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineercited by 56
  4. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  5. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 27
  6. 2024Privacy Preserving Prompt Engineering: A Surveycited by 24
  7. 2024DP-MLM: Differentially Private Text Rewriting Using Masked Language Modelscited by 24
  8. 2025Rethinking Data Protection in the (Generative) Artificial Intelligence Eracited by 17
  9. 2024Reconstruct Your Previous Conversations! Comprehensively Investigating Privacy Leakage Risks in Conversations with GPT Modelscited by 16
  10. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 15
  11. 2025Can Language Models Follow Multiple Turns of Entangled Instructions?cited by 14
  12. 2025POPri: Private Federated Learning using Preference-Optimized Synthetic Datacited by 14
38 more
  1. 2024The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Promptscited by 13
  2. 20241-Diffractor: Efficient and Utility-Preserving Text Obfuscation Leveraging Word-Level Metric Differential Privacycited by 10
  3. 2024Data-adaptive Differentially Private Prompt Synthesis for In-Context Learningcited by 10
  4. 2025InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacycited by 10
  5. 2025LLM-as-a-Judge for Privacy Evaluation? Exploring the Alignment of Human and LLM Perceptions of Privacy in Textual Datacited by 10
  6. 2024Just Rewrite It Again: A Post-Processing Method for Enhanced Semantic Similarity and Privacy Preservation of Differentially Private Rewritten Textcited by 9
  7. 2024NAP^2: A Benchmark for Naturalness and Privacy-Preserving Text Rewriting by Learning from Humancited by 8
  8. 2025On the Impact of Noise in Differentially Private Text Rewritingcited by 8
  9. 2024Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Modelscited by 6
  10. 2024Delving into Differentially Private Transformercited by 5
  11. 2024TAROT: Task-Oriented Authorship Obfuscation Using Policy Optimization Methodscited by 5
  12. 2025Spend Your Budget Wisely: Towards an Intelligent Distribution of the Privacy Budget in Differentially Private Text Rewritingcited by 5
  13. 2025DP-Fusion: Token-Level Differentially Private Inference for Large Language Modelscited by 5
  14. 2024Reconstruction of Differentially Private Text Sanitization via Large Language Modelscited by 4
  15. 2023ConfusionPrompt: Practical Private Inference for Online Large Language Modelscited by 3
  16. 2024Legal Fact Prediction: The Missing Piece in Legal Judgment Predictioncited by 3
  17. 2024CodeCipher: Learning to Obfuscate Source Code Against LLMscited by 3
  18. 2025Leveraging Semantic Triples for Private Document Generation with Local Differential Privacy Guaranteescited by 3
  19. 2025Zero-Shot Privacy-Aware Text Rewriting via Iterative Tree Searchcited by 3
  20. 2024A Collocation-based Method for Addressing Challenges in Word-level Metric Differential Privacycited by 2
  21. 2025DP-GTR: Differentially Private Prompt Protection via Group Text Rewritingcited by 2
  22. 2025Inspecting the Representation Manifold of Differentially-Private Textcited by 2
  23. 2025Anti-adversarial Learning: Desensitizing Prompts for Large Language Modelscited by 2
  24. 2025Privacy Preserving In-Context-Learning Framework for Large Language Modelscited by 2
  25. 2025Controlled Generation for Private Synthetic Textcited by 2
  26. 2026FedMosaic: Federated Retrieval-Augmented Generation via Parametric Adapterscited by 2
  27. 2026STAMP: Selective Task-Aware Mechanism for Text Privacycited by 2
  28. 2025Investigating User Perspectives on Differentially Private Text Privatizationcited by 1
  29. 2025The Double-edged Sword of LLM-based Data Reconstruction: Understanding and Mitigating Contextual Vulnerability in Word-level Differential Privacy Text Sanitizationcited by 1
  30. 2026Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimizationcited by 1
  31. 2026Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptationcited by 1
  32. 2026A Systematic Exploration of Text Decomposition and Budget Distribution in Differentially Private Text Obfuscationcited by 1
  33. 2024IDT: Dual-Task Adversarial Attacks for Privacy Protectioncited by 0
  34. 2025Stereo-LiDAR Fusion by Semi-Global Matching With Discrete Disparity-Matching Cost and Semidensificationcited by 0
  35. 2025Personalized Author Obfuscation with Large Language Modelscited by 0
  36. 2025When Explainability Meets Privacy: An Investigation at the Intersection of Post-hoc Explainability and Differential Privacy in the Context of Natural Language Processingcited by 0
  37. 2025RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewritingcited by 0
  38. 2025One Size Fits All? A Modular Adaptive Sanitization Kit (MASK) for Customizable Privacy-Preserving Phone Scam Detectioncited by 0

50 most cited of 63 All 63 (JSON)