mlsanityData to 7 Oct 2026

Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains

arXiv 2410.08327 ↗First posted 10 Oct 2024cs.CLcited by 12

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

The difficulty of anonymizing text data hinders the development and deployment of NLP in high-stakes domains that involve private data, such as healthcare and social services. Poorly anonymized sensitive data cannot be easily shared with annotators or external researchers, nor can it be used to train public models. In this work, we explore the feasibility of using synthetic data generated from differentially private language models in place of real data to facilitate the development of NLP in these domains without compromising privacy. In contrast to prior work, we generate synthetic data for real high-stakes domains, and we propose and conduct use-inspired evaluations to assess data quality. Our results show that prior simplistic evaluations have failed to highlight utility, privacy, and fairness issues in the synthetic data. Overall, our work underscores the need for further improvements to synthetic data generation for it to be a viable way to enable privacy-preserving data sharing.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 28 of its 53 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

28 of 53 references.

References · 28

  1. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  2. 2016Deep Learning with Differential Privacycited by 2,682
  3. 2018Hierarchical Neural Story Generationcited by 1,182
  4. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  5. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 408
  6. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  7. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  8. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  9. 2021Numerical Composition of Differential Privacycited by 174
  10. 2020How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligencecited by 128
  11. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 115
  12. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
16 more
  1. 2021Differential Privacy for Text Analytics via Natural Text Sanitizationcited by 74
  2. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  3. 2023Harnessing large-language models to generate private synthetic textcited by 65
  4. 2022The Limits of Word Level Differential Privacycited by 63
  5. 2021Selective Differential Privacy for Language Modelingcited by 61
  6. 2022A Customized Text Sanitization Mechanism with Differential Privacycited by 51
  7. 2021Coreference Resolution without Span Representationscited by 45
  8. 2020Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inferencecited by 43
  9. 2024Fine-Tuning Large Language Models with User-Level Differential Privacycited by 28
  10. 2020The Limits of Differential Privacy (and its Misuse in Data Release and Machine Learning)cited by 22
  11. 2024Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuningcited by 21
  12. 2023Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymizationcited by 18
  13. 2018An Empirical Evaluation of Deep Learning for ICD-9 Code Assignment using MIMIC-III Clinical Notescited by 16
  14. 2018Generation of Synthetic Electronic Medical Record Textcited by 14
  15. 2022Mention Annotations Alone Enable Efficient Domain Adaptation for Coreference Resolutioncited by 6
  16. 2023For Generated Text, Is NLI-Neutral Text the Best Text?cited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 5.

12 papers.

  1. 2025ALFA: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoningcited by 24
  2. 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 15
  3. 2025SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domainscited by 7
  4. 2025Struct-Bench: A Benchmark for Differentially Private Structured Text Generationcited by 7
  5. 2025How do we measure privacy in text? A survey of text anonymization metricscited by 5
  6. 2025Controlled Generation for Private Synthetic Textcited by 2
  7. 2025Privacy-Preserving Generation of Clinical Narratives from Medical Terminologiescited by 1
  8. 2026Leveraging Pretrained Language Models as Energy Functions for Glauber Dynamics Text Diffusioncited by 1
  9. 2026Measuring Privacy Risks and Tradeoffs in Financial Synthetic Data Generationcited by 0
  10. 2026The Conundrum of Trustworthy Research on Attacking Personally Identifiable Information Removal Techniquescited by 0
  11. 2026ContinuousBench: Can Differentially Private Synthetic Text Improve Capabilities?cited by 0
  12. 2026The Privacy-Hallucination Tradeoff in Differentially Private Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation
  2. 2024SafeSynthDP: Leveraging Large Language Models for Privacy-Preserving Synthetic Data Generation Using Differential Privacy
  3. 2024Private Synthetic Text Generation with Diffusion Models
  4. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data
  5. 2025Privacy-Preserving Synthetic Review Generation with Diverse Writing Styles Using LLMs
  6. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs
  7. 2025Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)
  8. 2025RewardDS: Privacy-Preserving Fine-Tuning for Large Language Models via Reward Driven Data Synthesis