# Evaluating Differentially Private Generation of Domain-Specific Text

arXiv: https://arxiv.org/abs/2508.20452 · HTML: https://mlsanity.com/paper/2508.20452/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Builds on (most cited first, 22 of 22)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Scalable Private Learning with PATE](https://mlsanity.com/paper/1802.08908/) (2018)
- [Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe](https://mlsanity.com/paper/2210.14348/) (2022)
- [Differentially Private Synthetic Data via Foundation Model APIs 2: Text](https://mlsanity.com/paper/2403.01749/) (2024)
- [Differentially Private Language Models for Secure Data Sharing](https://mlsanity.com/paper/2210.13918/) (2022)
- [The Limits of Word Level Differential Privacy](https://mlsanity.com/paper/2205.02130/) (2022)
- [Privacy-Preserving Retrieval-Augmented Generation with Differential Privacy](https://mlsanity.com/paper/2412.04697/) (2024)
- [TabSynDex: A Universal Metric for Robust Evaluation of Synthetic Tabular Data](https://mlsanity.com/paper/2207.05295/) (2022)
- [Training on the Benchmark Is Not All You Need](https://mlsanity.com/paper/2409.01790/) (2024)
- [Private prediction for large-scale synthetic text generation](https://mlsanity.com/paper/2407.12108/) (2024)
- [Differentially Private Next-Token Prediction of Large Language Models](https://mlsanity.com/paper/2403.15638/) (2024)
- [Synthetic is all you need: removing the auxiliary data assumption for membership inference attacks against synthetic data](https://mlsanity.com/paper/2307.01701/) (2023)
- [The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text](https://mlsanity.com/paper/2502.14921/) (2025)
- [Private Synthetic Text Generation with Diffusion Models](https://mlsanity.com/paper/2410.22971/) (2024)
- [A Two-Stage Decoder for Efficient ICD Coding](https://mlsanity.com/paper/2306.00005/) (2023)
- [LLMs are not Zero-Shot Reasoners for Biomedical Information Extraction](https://mlsanity.com/paper/2408.12249/) (2024)
- [MEDSAGE: Enhancing Robustness of Medical Dialogue Summarization to ASR Errors with LLM-generated Synthetic Dialogues](https://mlsanity.com/paper/2408.14418/) (2024)
- [Automated Refugee Case Analysis: An NLP Pipeline for Supporting Legal Practitioners](https://mlsanity.com/paper/2305.15533/) (2023)

## Built on by (most cited first, 5 of 5)

- [SynBench: A Benchmark for Differentially Private Text Generation](https://mlsanity.com/paper/2509.14594/) (2025)
- [Privacy-Preserving Generation of Clinical Narratives from Medical Terminologies](https://mlsanity.com/paper/2509.10882/) (2025)
- [ContinuousBench: Can Differentially Private Synthetic Text Improve Capabilities?](https://mlsanity.com/paper/2606.01849/) (2026)
- [Clinical Communication Processing with Models Trained on LLM-Generated Synthetic Data: A Structured Survey and Novel Application Case Studies](https://mlsanity.com/paper/2608.05993/) (2026)
- [PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatization](https://mlsanity.com/paper/2608.29624/) (2026)

## Abstract (arXiv)

Generative AI offers transformative potential for high-stakes domains such as healthcare and finance, yet privacy and regulatory barriers hinder the use of real-world data. To address this, differentially private synthetic data generation has emerged as a promising alternative. In this work, we introduce a unified benchmark to systematically evaluate the utility and fidelity of text datasets generated under formal Differential Privacy (DP) guarantees. Our benchmark addresses key challenges in domain-specific benchmarking, including choice of representative data and realistic privacy budgets, accounting for pre-training and a variety of evaluation metrics. We assess state-of-the-art privacy-preserving generation methods across five domain-specific datasets, revealing significant utility and fidelity degradation compared to real data, especially under strict privacy constraints. These findings underscore the limitations of current approaches, outline the need for advanced privacy-preserving data sharing methods and set a precedent regarding their evaluation in realistic scenarios.