# Differentially-private text generation degrades output language quality

arXiv: https://arxiv.org/abs/2509.11176 · HTML: https://mlsanity.com/paper/2509.11176/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 42 references that are in the map.
- First posted on arXiv 14 Sep 2025.

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models](https://mlsanity.com/paper/2210.08933/) (2022)
- [Will we run out of data? Limits of LLM scaling based on human-generated data](https://mlsanity.com/paper/2211.04325/) (2022)
- [Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe](https://mlsanity.com/paper/2210.14348/) (2022)
- [The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text](https://mlsanity.com/paper/2311.09807/) (2023)
- [Differentially Private Language Models for Secure Data Sharing](https://mlsanity.com/paper/2210.13918/) (2022)
- [The Limits of Word Level Differential Privacy](https://mlsanity.com/paper/2205.02130/) (2022)
- [Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scores](https://mlsanity.com/paper/2403.00553/) (2024)
- [Shifting AI Efficiency From Model-Centric to Data-Centric Compression](https://mlsanity.com/paper/2505.19147/) (2025)
- [Suicidal Ideation and Mental Disorder Detection with Attentive Relation Networks](https://mlsanity.com/paper/2004.07601/) (2020)
- [Private prediction for large-scale synthetic text generation](https://mlsanity.com/paper/2407.12108/) (2024)
- [New Alignment Methods for Discriminative Book Summarization](https://mlsanity.com/paper/1305.1319/) (2013)
- [There's No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction](https://mlsanity.com/paper/1610.02124/) (2016)
- [Private Synthetic Text Generation with Diffusion Models](https://mlsanity.com/paper/2410.22971/) (2024)
- [Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study](https://mlsanity.com/paper/1909.06618/) (2019)
- [Differentially Private Learning Needs Better Model Initialization and Self-Distillation](https://mlsanity.com/paper/2410.17566/) (2024)
- [De-amplifying Bias from Differential Privacy in Language Model Fine-tuning](https://mlsanity.com/paper/2402.04489/) (2024)
- [Does Differential Privacy Impact Bias in Pretrained NLP Models?](https://mlsanity.com/paper/2410.18749/) (2024)

## Built on by (most cited first, 1 of 1)

- [Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimization](https://mlsanity.com/paper/2602.20743/) (2026)

## Abstract (arXiv)

Ensuring user privacy by synthesizing data from large language models (LLMs) tuned under differential privacy (DP) has become popular recently. However, the impact of DP fine-tuned LLMs on the quality of the language and the utility of the texts they produce has not been investigated. In this work, we tune five LLMs with three corpora under four levels of privacy and assess the length, the grammatical correctness, and the lexical diversity of the text outputs they produce. We also probe the utility of the synthetic outputs in downstream classification tasks such as book genre recognition based on book descriptions and cause of death recognition based on verbal autopsies. The results indicate that LLMs tuned under stronger privacy constrains produce texts that are shorter by at least 77 %, that are less grammatically correct by at least 9 %, and are less diverse by at least 10 % in bi-gram diversity. Furthermore, the accuracy they reach in downstream classification tasks decreases, which might be detrimental to the usefulness of the generated synthetic data.