# DP-BART for Privatized Text Rewriting under Local Differential Privacy

arXiv: https://arxiv.org/abs/2302.07636 · HTML: https://mlsanity.com/paper/2302.07636/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 15 Feb 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces](https://mlsanity.com/paper/1805.10190/) (2018)
- [Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising](https://mlsanity.com/paper/1805.06530/) (2018)
- [Differentially Private Language Models for Secure Data Sharing](https://mlsanity.com/paper/2210.13918/) (2022)
- [The Limits of Word Level Differential Privacy](https://mlsanity.com/paper/2205.02130/) (2022)
- [Leveraging Hierarchical Representations for Preserving Privacy and Utility in Text](https://mlsanity.com/paper/1910.08917/) (2019)
- [A Differentially Private Text Perturbation Method Using a Regularized Mahalanobis Metric](https://mlsanity.com/paper/2010.11947/) (2020)
- [Differential Privacy: An Economic Method for Choosing Epsilon](https://mlsanity.com/paper/1402.3329/) (2014)
- [ER-AE: Differentially Private Text Generation for Authorship Anonymization](https://mlsanity.com/paper/1907.08736/) (2019)
- [ADePT: Auto-encoder based Differentially Private Text Transformation](https://mlsanity.com/paper/2102.01502/) (2021)
- [DP-Rewrite: Towards Reproducibility and Transparency in Differentially Private Text Rewriting](https://mlsanity.com/paper/2208.10400/) (2022)
- [A Comprehensive Survey on Local Differential Privacy Toward Data Statistics and Analysis](https://mlsanity.com/paper/2010.05253/) (2020)
- [When differential privacy meets NLP: The devil is in the detail](https://mlsanity.com/paper/2109.03175/) (2021)
- [Privacy-Preserving Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/2102.09604/) (2021)
- [Differential Privacy in Natural Language Processing: The Story So Far](https://mlsanity.com/paper/2208.08140/) (2022)
- [How reparametrization trick broke differentially-private text representation learning](https://mlsanity.com/paper/2202.12138/) (2022)

## Built on by (most cited first, 28 of 28)

- [Safeguarding Large Language Models: A Survey](https://mlsanity.com/paper/2406.02622/) (2024)
- [Building Guardrails for Large Language Models](https://mlsanity.com/paper/2402.01822/) (2024)
- [Privacy in Large Language Models: Attacks, Defenses and Future Directions](https://mlsanity.com/paper/2310.10383/) (2023)
- [IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomization](https://mlsanity.com/paper/2407.02956/) (2024)
- [A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage](https://mlsanity.com/paper/2504.21035/) (2025)
- [Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions](https://mlsanity.com/paper/2412.16504/) (2024)
- [Just Rewrite It Again: A Post-Processing Method for Enhanced Semantic Similarity and Privacy Preservation of Differentially Private Rewritten Text](https://mlsanity.com/paper/2405.19831/) (2024)
- [Neural Text Sanitization with Privacy Risk Indicators: An Empirical Analysis](https://mlsanity.com/paper/2310.14312/) (2023)
- [Spend Your Budget Wisely: Towards an Intelligent Distribution of the Privacy Budget in Differentially Private Text Rewriting](https://mlsanity.com/paper/2503.22379/) (2025)
- [A Survey on Current Trends and Recent Advances in Text Anonymization](https://mlsanity.com/paper/2508.21587/) (2025)
- [ObfuscaTune: Obfuscated Offsite Fine-tuning and Inference of Proprietary LLMs on Private Datasets](https://mlsanity.com/paper/2407.02960/) (2024)
- [Reconstruction of Differentially Private Text Sanitization via Large Language Models](https://mlsanity.com/paper/2410.12443/) (2024)
- [ExpShield: Safeguarding Web Text from Unauthorized Crawling and LLM Exploitation](https://mlsanity.com/paper/2412.21123/) (2024)
- [MaskSQL: Safeguarding Privacy for LLM-Based Text-to-SQL via Abstraction](https://mlsanity.com/paper/2509.23459/) (2025)
- [Federated Domain-Specific Knowledge Transfer on Large Language Models Using Synthetic Data](https://mlsanity.com/paper/2405.14212/) (2024)
- [Mitigating Gradient Inversion Risks in Language Models via Token Obfuscation](https://mlsanity.com/paper/2602.15897/) (2026)
- [Defending Against Authorship Identification Attacks](https://mlsanity.com/paper/2310.01568/) (2023)
- [Empirical Calibration and Metric Differential Privacy in Language Models](https://mlsanity.com/paper/2503.13872/) (2025)
- [The Double-edged Sword of LLM-based Data Reconstruction: Understanding and Mitigating Contextual Vulnerability in Word-level Differential Privacy Text Sanitization](https://mlsanity.com/paper/2508.18976/) (2025)
- [Towards Privacy-Preserving Machine Translation at the Inference Stage: A New Task and Benchmark](https://mlsanity.com/paper/2603.14756/) (2026)
- [Preempting Text Sanitization Utility in Resource-Constrained Privacy-Preserving LLM Interactions](https://mlsanity.com/paper/2411.11521/) (2024)
- [Stereo-LiDAR Fusion by Semi-Global Matching With Discrete Disparity-Matching Cost and Semidensification](https://mlsanity.com/paper/2504.05148/) (2025)
- [RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewriting](https://mlsanity.com/paper/2508.19286/) (2025)
- [CTIGuardian: A Few-Shot Framework for Mitigating Privacy Leakage in Fine-Tuned LLMs](https://mlsanity.com/paper/2512.12914/) (2025)
- [Learning Demographic-Conditioned Mobility Trajectories with Aggregate Supervision](https://mlsanity.com/paper/2603.03275/) (2026)
- [LLM Anonymization Against Agentic Re-Identification](https://mlsanity.com/paper/2605.30848/) (2026)
- [Differentially Private Datastore Generation for Retrieval-Augmented Inference](https://mlsanity.com/paper/2606.01413/) (2026)
- [SpliTEE: Fast and Private LLM Inference by Coupling GPU-Assisted Trusted Execution Environments with Differential Privacy](https://mlsanity.com/paper/2609.15039/) (2026)

## Abstract (arXiv)

Privatized text rewriting with local differential privacy (LDP) is a recent approach that enables sharing of sensitive textual documents while formally guaranteeing privacy protection to individuals. However, existing systems face several issues, such as formal mathematical flaws, unrealistic privacy guarantees, privatization of only individual words, as well as a lack of transparency and reproducibility. In this paper, we propose a new system 'DP-BART' that largely outperforms existing LDP systems. Our approach uses a novel clipping method, iterative pruning, and further training of internal representations which drastically reduces the amount of noise required for DP guarantees. We run experiments on five textual datasets of varying sizes, rewriting them at different privacy guarantees and evaluating the rewritten texts on downstream text classification tasks. Finally, we thoroughly discuss the privatized text rewriting approach and its limitations, including the problem of the strict text adjacency constraint in the LDP paradigm that leads to the high noise requirement.