# TEM: High Utility Metric Differential Privacy on Text

arXiv: https://arxiv.org/abs/2107.07928 · HTML: https://mlsanity.com/paper/2107.07928/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2021.

## Built on by (most cited first, 30 of 43)

- [The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)](https://mlsanity.com/paper/2402.16893/) (2024)
- [Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation](https://mlsanity.com/paper/2309.11765/) (2023)
- [Differentially Private Synthetic Data via Foundation Model APIs 2: Text](https://mlsanity.com/paper/2403.01749/) (2024)
- [Locally Differentially Private Document Generation Using Zero Shot Prompting](https://mlsanity.com/paper/2310.16111/) (2023)
- [DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer](https://mlsanity.com/paper/2312.03724/) (2023)
- [Privacy Preserving Prompt Engineering: A Survey](https://mlsanity.com/paper/2404.06001/) (2024)
- [DP-MLM: Differentially Private Text Rewriting Using Masked Language Models](https://mlsanity.com/paper/2407.00637/) (2024)
- [Differentially Private Natural Language Models: Recent Advances and Future Directions](https://mlsanity.com/paper/2301.09112/) (2023)
- [Sentence-level Privacy for Document Embeddings](https://mlsanity.com/paper/2205.04605/) (2022)
- [Reconstruct Your Previous Conversations! Comprehensively Investigating Privacy Leakage Risks in Conversations with GPT Models](https://mlsanity.com/paper/2402.02987/) (2024)
- [Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs](https://mlsanity.com/paper/2503.12347/) (2025)
- [Guiding Text-to-Text Privatization by Syntax](https://mlsanity.com/paper/2306.01471/) (2023)
- [Driving Context into Text-to-Text Privatization](https://mlsanity.com/paper/2306.01457/) (2023)
- [A Comparative Analysis of Word-Level Metric Differential Privacy: Benchmarking The Privacy-Utility Trade-off](https://mlsanity.com/paper/2404.03324/) (2024)
- [You Are What You Write: Preserving Privacy in the Era of Large Language Models](https://mlsanity.com/paper/2204.09391/) (2022)
- [Differential Privacy in Natural Language Processing: The Story So Far](https://mlsanity.com/paper/2208.08140/) (2022)
- [1-Diffractor: Efficient and Utility-Preserving Text Obfuscation Leveraging Word-Level Metric Differential Privacy](https://mlsanity.com/paper/2405.01678/) (2024)
- [InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacy](https://mlsanity.com/paper/2507.02974/) (2025)
- [Just Rewrite It Again: A Post-Processing Method for Enhanced Semantic Similarity and Privacy Preservation of Differentially Private Rewritten Text](https://mlsanity.com/paper/2405.19831/) (2024)
- [On the Impact of Noise in Differentially Private Text Rewriting](https://mlsanity.com/paper/2501.19022/) (2025)
- [Delving into Differentially Private Transformer](https://mlsanity.com/paper/2405.18194/) (2024)
- [Spend Your Budget Wisely: Towards an Intelligent Distribution of the Privacy Budget in Differentially Private Text Rewriting](https://mlsanity.com/paper/2503.22379/) (2025)
- [Reconstruction of Differentially Private Text Sanitization via Large Language Models](https://mlsanity.com/paper/2410.12443/) (2024)
- [Private Language Models via Truncated Laplacian Mechanism](https://mlsanity.com/paper/2410.08027/) (2024)
- [Leveraging Semantic Triples for Private Document Generation with Local Differential Privacy Guarantees](https://mlsanity.com/paper/2508.20736/) (2025)
- [A Collocation-based Method for Addressing Challenges in Word-level Metric Differential Privacy](https://mlsanity.com/paper/2407.00638/) (2024)
- [DP-GTR: Differentially Private Prompt Protection via Group Text Rewriting](https://mlsanity.com/paper/2503.04990/) (2025)
- [Inspecting the Representation Manifold of Differentially-Private Text](https://mlsanity.com/paper/2503.14991/) (2025)
- [Privacy Preserving In-Context-Learning Framework for Large Language Models](https://mlsanity.com/paper/2509.13625/) (2025)
- [STAMP: Selective Task-Aware Mechanism for Text Privacy](https://mlsanity.com/paper/2603.12237/) (2026)

## Abstract (arXiv)

Ensuring the privacy of users whose data are used to train Natural Language Processing (NLP) models is necessary to build and maintain customer trust. Differential Privacy (DP) has emerged as the most successful method to protect the privacy of individuals. However, applying DP to the NLP domain comes with unique challenges. The most successful previous methods use a generalization of DP for metric spaces, and apply the privatization by adding noise to inputs in the metric space of word embeddings. However, these methods assume that one specific distance measure is being used, ignore the density of the space around the input, and assume the embeddings used have been trained on non-sensitive data. In this work we propose Truncated Exponential Mechanism (TEM), a general method that allows the privatization of words using any distance metric, on embeddings that can be trained on sensitive data. Our method makes use of the exponential mechanism to turn the privatization step into a selection problem. This allows the noise applied to be calibrated to the density of the embedding space around the input, and makes domain adaptation possible for the embeddings. In our experiments, we demonstrate that our method significantly outperforms the state-of-the-art in terms of utility for the same level of privacy, while providing more flexibility in the metric selection.