mlsanityData to 7 Oct 2026

The Fire Thief Is Also the Keeper: Balancing Usability and Privacy in Prompts

arXiv 2406.14318 ↗First posted 20 Jun 2024cs.CR cs.AI cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

The rapid adoption of online chatbots represents a significant advancement in artificial intelligence. However, this convenience brings considerable privacy concerns, as prompts can inadvertently contain sensitive information exposed to large language models (LLMs). Limited by high computational costs, reduced task usability, and excessive system modifications, previous works based on local deployment, embedding perturbation, and homomorphic encryption are inapplicable to online prompt-based LLM applications. To address these issues, this paper introduces Prompt Privacy Sanitizer (i.e., ProSan), an end-to-end prompt privacy protection framework that can produce anonymized prompts with contextual privacy removed while maintaining task usability and human readability. It can also be seamlessly integrated into the online LLM service pipeline. To achieve high usability and dynamic anonymity, ProSan flexibly adjusts its protection targets and strength based on the importance of the words and the privacy leakage risk of the prompts. Additionally, ProSan is capable of adapting to diverse computational resource conditions, ensuring privacy protection even for mobile devices with limited computing power. Our experiments demonstrate that ProSan effectively removes private information across various tasks, including question answering, text summarization, and code generation, with minimal reduction in task performance.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 49 of its 73 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

49 of 73 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  10. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
37 more
  1. 2016Deep Learning with Differential Privacycited by 2,682
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2018Deep contextualized word representationscited by 2,285
  5. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  6. 2021Membership Inference Attacks From First Principlescited by 839
  7. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 530
  8. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  9. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  10. 2018Scalable Private Learning with PATEcited by 266
  11. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 218
  12. 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 185
  13. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 115
  14. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  15. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 94
  16. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 80
  17. 2021Differential Privacy for Text Analytics via Natural Text Sanitizationcited by 74
  18. 2022Recovering Private Text in Federated Learning of Language Modelscited by 67
  19. 2023DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Passcited by 65
  20. 2022The Limits of Word Level Differential Privacycited by 63
  21. 2023Locally Differentially Private Document Generation Using Zero Shot Promptingcited by 63
  22. 2021Natural Language Understanding with Privacy-Preserving BERTcited by 54
  23. 2019Leveraging Hierarchical Representations for Preserving Privacy and Utility in Textcited by 49
  24. 2023Reducing Privacy Risks in Online Self-Disclosures with Language Modelscited by 47
  25. 2022A Survey on Model Compression and Acceleration for Pretrained Language Modelscited by 45
  26. 2017A4NT: Author Attribute Anonymity by Adversarial Training of Neural Machine Translationcited by 41
  27. 2023Hide and Seek (HaS): A Lightweight Framework for Prompt Privacy Protectioncited by 37
  28. 2020TextHide: Tackling Data Privacy in Language Understanding Taskscited by 32
  29. 2024CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Followingcited by 32
  30. 2021ADePT: Auto-encoder based Differentially Private Text Transformationcited by 31
  31. 2020Towards Differentially Private Text Representationscited by 28
  32. 2024Privacy Preserving Prompt Engineering: A Surveycited by 24
  33. 2023Protecting User Privacy in Remote Conversational Systems: A Privacy-Preserving framework based on text sanitizationcited by 23
  34. 2021CAPE: Context-Aware Private Embeddings for Private Language Learningcited by 22
  35. 2022EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generationcited by 21
  36. 2024Reconstruct Your Previous Conversations! Comprehensively Investigating Privacy Leakage Risks in Conversations with GPT Modelscited by 16
  37. 2022Sentence-level Privacy for Document Embeddingscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 6.

13 papers.

Sort:

By area:

  1. 2024Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generationcited by 11
  2. 2024Casper: Prompt Sanitization for Protecting User Privacy in Web-Based Large Language Modelscited by 8
  3. 2025PII-Bench: Evaluating Query-Aware Privacy Protection Systemscited by 7
  4. 2025LLM Access Shield: Domain-Specific LLM Framework for Privacy Policy Compliancecited by 6
  5. 2026Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)cited by 2
  6. 2024Confidential Prompting: Privacy-preserving LLM Inference on Cloudcited by 1
  7. 2026Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptationcited by 1
  8. 2026PrivScope: Task-scoped Disclosure Control for Hybrid Agentic Systemscited by 1
  9. 2026Demystifying the Privacy-Utility Trade-off in LLM Interactionscited by 1
  10. 2024Preempting Text Sanitization Utility in Resource-Constrained Privacy-Preserving LLM Interactionscited by 0
  11. 2025Stereo-LiDAR Fusion by Semi-Global Matching With Discrete Disparity-Matching Cost and Semidensificationcited by 0
  12. 2026PromptGraph: Graph-Guided Prompt Sanitization for Balancing Privacy and Utility in LLM Inferencecited by 0
1 more
  1. 2026LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communicationcited by 0