# Prεεmpt: Sanitizing Sensitive Prompts for LLMs

arXiv: https://arxiv.org/abs/2504.05147 · HTML: https://mlsanity.com/paper/2504.05147/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 80 references that are in the map.
- First posted on arXiv 7 Apr 2025.

## Builds on (most cited first, 3 of 3)

- [A Survey of Optimization Methods from a Machine Learning Perspective](https://mlsanity.com/paper/1906.06821/) (2019)
- [ParamILS: An Automatic Algorithm Configuration Framework](https://mlsanity.com/paper/1401.3492/) (2014)
- [Assessing Ranking and Effectiveness of Evolutionary Algorithm Hyperparameters Using Global Sensitivity Analysis Methodologies](https://mlsanity.com/paper/2207.04820/) (2022)

## Built on by (most cited first, 12 of 12)

- [MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation](https://mlsanity.com/paper/2510.15186/) (2025)
- [Operationalizing Data Minimization for Privacy-Preserving LLM Prompting](https://mlsanity.com/paper/2510.03662/) (2025)
- [PPMI: Privacy-Preserving LLM Interaction with Socratic Chain-of-Thought Reasoning and Homomorphically Encrypted Vector Databases](https://mlsanity.com/paper/2506.17336/) (2025)
- [Can Large Language Models Really Recognize Your Name?](https://mlsanity.com/paper/2505.14549/) (2025)
- [MaskSQL: Safeguarding Privacy for LLM-Based Text-to-SQL via Abstraction](https://mlsanity.com/paper/2509.23459/) (2025)
- [PAAC: Privacy-Aware Agentic Device-Cloud Collaboration](https://mlsanity.com/paper/2605.08646/) (2026)
- [Beyond Direct Identifiers: Probabilistic Privacy Risk Estimation for Privacy-Conscious LLM Query Delegation](https://mlsanity.com/paper/2608.09140/) (2026)
- [Demystifying the Privacy-Utility Trade-off in LLM Interactions](https://mlsanity.com/paper/2609.10992/) (2026)
- [DP-MGTD: Privacy-Preserving Machine-Generated Text Detection via Adaptive Differentially Private Entity Sanitization](https://mlsanity.com/paper/2601.04641/) (2026)
- [Selective Token-Level Cryptographic Redaction for Privacy-Preserving Clinical Deployment of Large Language Models](https://mlsanity.com/paper/2606.03399/) (2026)
- [PromptPET: Privacy-Utility Optimized Prompt Obfuscation](https://mlsanity.com/paper/2607.02932/) (2026)
- [PolicyGuard: Prompt-Configurable Semantic DLP for LLM Coding Agents](https://mlsanity.com/paper/2608.02687/) (2026)

## Abstract (arXiv)

The rise of large language models (LLMs) has introduced new privacy challenges, particularly during inference where sensitive information in prompts may be exposed to proprietary LLM APIs. In this paper, we address the problem of formally protecting the sensitive information contained in a prompt while maintaining response quality. To this end, first, we introduce a cryptographically inspired notion of a prompt sanitizer which transforms an input prompt to protect its sensitive tokens. Second, we propose Prεεmpt, a novel system that implements a prompt sanitizer. Prεεmpt categorizes sensitive tokens into two types: (1) those where the LLM's response depends solely on the format (such as SSNs, credit card numbers), for which we use format-preserving encryption (FPE); and (2) those where the response depends on specific values, (such as age, salary) for which we apply metric differential privacy (mDP). Our evaluation demonstrates that Prεεmpt is a practical method to achieve meaningful privacy guarantees, while maintaining high utility compared to unsanitized prompts, and outperforming prior methods