# Guiding Text-to-Text Privatization by Syntax

arXiv: https://arxiv.org/abs/2306.01471 · HTML: https://mlsanity.com/paper/2306.01471/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Information Leakage in Embedding Models](https://mlsanity.com/paper/2004.00053/) (2020)
- [Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations](https://mlsanity.com/paper/1910.08902/) (2019)
- [The Limits of Word Level Differential Privacy](https://mlsanity.com/paper/2205.02130/) (2022)
- [Privacy-preserving Neural Representations of Text](https://mlsanity.com/paper/1808.09408/) (2018)
- [Generalised Differential Privacy for Text Document Processing](https://mlsanity.com/paper/1811.10256/) (2018)
- [Leveraging Hierarchical Representations for Preserving Privacy and Utility in Text](https://mlsanity.com/paper/1910.08917/) (2019)
- [TEM: High Utility Metric Differential Privacy on Text](https://mlsanity.com/paper/2107.07928/) (2021)
- [A Differentially Private Text Perturbation Method Using a Regularized Mahalanobis Metric](https://mlsanity.com/paper/2010.11947/) (2020)
- [C-sanitized: a privacy model for document redaction and sanitization](https://mlsanity.com/paper/1406.4285/) (2014)
- [SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Mining](https://mlsanity.com/paper/1805.00904/) (2018)
- [On a Utilitarian Approach to Privacy Preserving Text Generation](https://mlsanity.com/paper/2104.11838/) (2021)
- [BRR: Preserving Privacy of Text Data Efficiently on Device](https://mlsanity.com/paper/2107.07923/) (2021)
- [dpUGC: Learn Differentially Private Representation for User Generated Contents](https://mlsanity.com/paper/1903.10453/) (2019)
- [Author Obfuscation Using Generalised Differential Privacy](https://mlsanity.com/paper/1805.08866/) (2018)

## Built on by (most cited first, 14 of 14)

- [Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation](https://mlsanity.com/paper/2309.11765/) (2023)
- [Data-adaptive Differentially Private Prompt Synthesis for In-Context Learning](https://mlsanity.com/paper/2410.12085/) (2024)
- [Spend Your Budget Wisely: Towards an Intelligent Distribution of the Privacy Budget in Differentially Private Text Rewriting](https://mlsanity.com/paper/2503.22379/) (2025)
- [How do we measure privacy in text? A survey of text anonymization metrics](https://mlsanity.com/paper/2512.01109/) (2025)
- [Private Language Models via Truncated Laplacian Mechanism](https://mlsanity.com/paper/2410.08027/) (2024)
- [Leveraging Semantic Triples for Private Document Generation with Local Differential Privacy Guarantees](https://mlsanity.com/paper/2508.20736/) (2025)
- [Investigating User Perspectives on Differentially Private Text Privatization](https://mlsanity.com/paper/2503.09338/) (2025)
- [The Double-edged Sword of LLM-based Data Reconstruction: Understanding and Mitigating Contextual Vulnerability in Word-level Differential Privacy Text Sanitization](https://mlsanity.com/paper/2508.18976/) (2025)
- [A Systematic Exploration of Text Decomposition and Budget Distribution in Differentially Private Text Obfuscation](https://mlsanity.com/paper/2605.01065/) (2026)
- [Disentangling the Linguistic Competence of Privacy-Preserving BERT](https://mlsanity.com/paper/2310.11363/) (2023)
- [Stereo-LiDAR Fusion by Semi-Global Matching With Discrete Disparity-Matching Cost and Semidensification](https://mlsanity.com/paper/2504.05148/) (2025)
- [One Size Fits All? A Modular Adaptive Sanitization Kit (MASK) for Customizable Privacy-Preserving Phone Scam Detection](https://mlsanity.com/paper/2510.18493/) (2025)
- [Differentially-Private Text Rewriting reshapes Linguistic Style](https://mlsanity.com/paper/2604.26656/) (2026)
- [PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatization](https://mlsanity.com/paper/2608.29624/) (2026)

## Abstract (arXiv)

Metric Differential Privacy is a generalization of differential privacy tailored to address the unique challenges of text-to-text privatization. By adding noise to the representation of words in the geometric space of embeddings, words are replaced with words located in the proximity of the noisy representation. Since embeddings are trained based on word co-occurrences, this mechanism ensures that substitutions stem from a common semantic context. Without considering the grammatical category of words, however, this mechanism cannot guarantee that substitutions play similar syntactic roles. We analyze the capability of text-to-text privatization to preserve the grammatical category of words after substitution and find that surrogate texts consist almost exclusively of nouns. Lacking the capability to produce surrogate texts that correlate with the structure of the sensitive texts, we encompass our analysis by transforming the privatization step into a candidate selection problem in which substitutions are directed to words with matching grammatical properties. We demonstrate a substantial improvement in the performance of downstream tasks by up to 4.66% while retaining comparative privacy guarantees.