# Spend Your Budget Wisely: Towards an Intelligent Distribution of the Privacy Budget in Differentially Private Text Rewriting

arXiv: https://arxiv.org/abs/2503.22379 · HTML: https://mlsanity.com/paper/2503.22379/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 28 Mar 2025.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Scalable Extraction of Training Data from (Production) Language Models](https://mlsanity.com/paper/2311.17035/) (2023)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy](https://mlsanity.com/paper/2307.00691/) (2023)
- [Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations](https://mlsanity.com/paper/1910.08902/) (2019)
- [On Protecting the Data Privacy of Large Language Models (LLMs): A Survey](https://mlsanity.com/paper/2403.05156/) (2024)
- [Differential Privacy for Text Analytics via Natural Text Sanitization](https://mlsanity.com/paper/2106.01221/) (2021)
- [DocNLI: A Large-scale Dataset for Document-level Natural Language Inference](https://mlsanity.com/paper/2106.09449/) (2021)
- [Locally Differentially Private Document Generation Using Zero Shot Prompting](https://mlsanity.com/paper/2310.16111/) (2023)
- [Generalised Differential Privacy for Text Document Processing](https://mlsanity.com/paper/1811.10256/) (2018)
- [A Customized Text Sanitization Mechanism with Differential Privacy](https://mlsanity.com/paper/2207.01193/) (2022)
- [TEM: High Utility Metric Differential Privacy on Text](https://mlsanity.com/paper/2107.07928/) (2021)
- [ER-AE: Differentially Private Text Generation for Authorship Anonymization](https://mlsanity.com/paper/1907.08736/) (2019)
- [DP-Rewrite: Towards Reproducibility and Transparency in Differentially Private Text Rewriting](https://mlsanity.com/paper/2208.10400/) (2022)
- [DP-BART for Privatized Text Rewriting under Local Differential Privacy](https://mlsanity.com/paper/2302.07636/) (2023)
- [When differential privacy meets NLP: The devil is in the detail](https://mlsanity.com/paper/2109.03175/) (2021)
- [SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Mining](https://mlsanity.com/paper/1805.00904/) (2018)
- [Differentially Private Knowledge Distillation via Synthetic Text Generation](https://mlsanity.com/paper/2403.00932/) (2024)
- [Sentence-level Privacy for Document Embeddings](https://mlsanity.com/paper/2205.04605/) (2022)
- [Guiding Text-to-Text Privatization by Syntax](https://mlsanity.com/paper/2306.01471/) (2023)
- [Thinking Outside of the Differential Privacy Box: A Case Study in Text Privatization with Language Model Prompting](https://mlsanity.com/paper/2410.00751/) (2024)
- [1-Diffractor: Efficient and Utility-Preserving Text Obfuscation Leveraging Word-Level Metric Differential Privacy](https://mlsanity.com/paper/2405.01678/) (2024)
- [Research Challenges in Designing Differentially Private Text Generation Mechanisms](https://mlsanity.com/paper/2012.05403/) (2020)
- [DP-NMT: Scalable Differentially-Private Machine Translation](https://mlsanity.com/paper/2311.14465/) (2023)

## Built on by (most cited first, 5 of 5)

- [STAMP: Selective Task-Aware Mechanism for Text Privacy](https://mlsanity.com/paper/2603.12237/) (2026)
- [The Double-edged Sword of LLM-based Data Reconstruction: Understanding and Mitigating Contextual Vulnerability in Word-level Differential Privacy Text Sanitization](https://mlsanity.com/paper/2508.18976/) (2025)
- [A Systematic Exploration of Text Decomposition and Budget Distribution in Differentially Private Text Obfuscation](https://mlsanity.com/paper/2605.01065/) (2026)
- [PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatization](https://mlsanity.com/paper/2608.29624/) (2026)
- [DP-IPI: A Hybrid Differential Privacy Text Rewriting Mechanism for Indirect Personal Identifiers in Clinical Texts](https://mlsanity.com/paper/2609.29684/) (2026)

## Abstract (arXiv)

The task of textitDifferentially Private Text Rewriting is a class of text privatization techniques in which (sensitive) input textual documents are textitrewritten under Differential Privacy (DP) guarantees. The motivation behind such methods is to hide both explicit and implicit identifiers that could be contained in text, while still retaining the semantic meaning of the original text, thus preserving utility. Recent years have seen an uptick in research output in this field, offering a diverse array of word-, sentence-, and document-level DP rewriting methods. Common to these methods is the selection of a privacy budget (i.e., the varepsilon parameter), which governs the degree to which a text is privatized. One major limitation of previous works, stemming directly from the unique structure of language itself, is the lack of consideration of textitwhere the privacy budget should be allocated, as not all aspects of language, and therefore text, are equally sensitive or personal. In this work, we are the first to address this shortcoming, asking the question of how a given privacy budget can be intelligently and sensibly distributed amongst a target document. We construct and evaluate a toolkit of linguistics- and NLP-based methods used to allocate a privacy budget to constituent tokens in a text document. In a series of privacy and utility experiments, we empirically demonstrate that given the same privacy budget, intelligent distribution leads to higher privacy levels and more positive trade-offs than a naive distribution of varepsilon. Our work highlights the intricacies of text privatization with DP, and furthermore, it calls for further work on finding more efficient ways to maximize the privatization benefits offered by DP in text rewriting.