# SafeSynthDP: Leveraging Large Language Models for Privacy-Preserving Synthetic Data Generation Using Differential Privacy

arXiv: https://arxiv.org/abs/2412.20641 · HTML: https://mlsanity.com/paper/2412.20641/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 7 of its 40 references that are in the map.
- First posted on arXiv 30 Dec 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Privacy-preserving tabular data synthesis](https://mlsanity.com/synth/privacy-preserving-tabular-data-synthesis/)

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Best Practices and Lessons Learned on Synthetic Data](https://mlsanity.com/paper/2404.07503/) (2024)
- [Differentially Private Synthetic Data: Applied Evaluations and Enhancements](https://mlsanity.com/paper/2011.05537/) (2020)

## Built on by (most cited first, 5 of 5)

- [SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domains](https://mlsanity.com/paper/2507.07229/) (2025)
- [Struct-Bench: A Benchmark for Differentially Private Structured Text Generation](https://mlsanity.com/paper/2509.10696/) (2025)
- [Towards Privacy-aware Mental Health AI Models: Advances, Challenges, and Opportunities](https://mlsanity.com/paper/2502.00451/) (2025)
- [Patient-Zero: Scaling Synthetic Patient Agents to Real-World Distributions without Real Patient Data](https://mlsanity.com/paper/2509.11078/) (2025)
- [From Fragmentation to Integration: Exploring the Design Space of AI Agents for Human-as-the-Unit Privacy Management](https://mlsanity.com/paper/2602.05016/) (2026)

## Abstract (arXiv)

Machine learning (ML) models frequently rely on training data that may include sensitive or personal information, raising substantial privacy concerns. Legislative frameworks such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) have necessitated the development of strategies that preserve privacy while maintaining the utility of data. In this paper, we investigate the capability of Large Language Models (LLMs) to generate synthetic datasets integrated with Differential Privacy (DP) mechanisms, thereby enabling data-driven research and model training without direct exposure of sensitive information. Our approach incorporates DP-based noise injection methods, including Laplace and Gaussian distributions, into the data generation process. We then evaluate the utility of these DP-enhanced synthetic datasets by comparing the performance of ML models trained on them against models trained on the original data. To substantiate privacy guarantees, we assess the resilience of the generated synthetic data to membership inference attacks and related threats. The experimental results demonstrate that integrating DP within LLM-driven synthetic data generation offers a viable balance between privacy protection and data utility. This study provides a foundational methodology and insight into the privacy-preserving capabilities of LLMs, paving the way for compliant and effective ML research and applications.