mlsanityData to 7 Oct 2026

Differentially Private Tabular Data Synthesis using Large Language Models

arXiv 2406.01457 ↗First posted 3 Jun 2024cs.LG cs.CL cs.CRcited by 16

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

Synthetic tabular data generation with differential privacy is a crucial problem to enable data sharing with formal privacy. Despite a rich history of methodological research and development, developing differentially private tabular data generators that can provide realistic synthetic datasets remains challenging. This paper introduces DP-LLMTGen -- a novel framework for differentially private tabular data synthesis that leverages pretrained large language models (LLMs). DP-LLMTGen models sensitive datasets using a two-stage fine-tuning procedure with a novel loss function specifically designed for tabular data. Subsequently, it generates synthetic data through sampling the fine-tuned LLMs. Our empirical evaluation demonstrates that DP-LLMTGen outperforms a variety of existing mechanisms across multiple datasets and privacy settings. Additionally, we conduct an ablation study and several experimental analyses to deepen our understanding of LLMs in addressing this important problem. Finally, we highlight the controllable generation ability of DP-LLMTGen through a fairness-constrained generation setting.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 40 of its 57 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

40 of 57 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2016Deep Learning with Differential Privacycited by 2,682
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  8. 2019Modeling Tabular data using Conditional GANcited by 695
  9. 2021Tabular Data: Deep Learning is Not All You Needcited by 530
  10. 2021Deep Neural Networks and Tabular Data: A Surveycited by 489
  11. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  12. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 375
28 more
  1. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 373
  2. 2022Language Models are Realistic Tabular Data Generatorscited by 290
  3. 2022TabLLM: Few-shot Classification of Tabular Data with Large Language Modelscited by 273
  4. 2018Scalable Private Learning with PATEcited by 266
  5. 2018Differentially Private Generative Adversarial Networkcited by 235
  6. 2018Data Synthesis based on Generative Adversarial Networkscited by 184
  7. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  8. 2020DP-CGAN: Differentially Private Synthetic Data and Label Generationcited by 118
  9. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  10. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitationscited by 96
  11. 2022AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Datacited by 88
  12. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  13. 2023Harnessing large-language models to generate private synthetic textcited by 65
  14. 2023Differentially Private Synthetic Data via Foundation Model APIs 1: Imagescited by 63
  15. 2021Iterative Methods for Private Synthetic Data: Unifying Framework and New Methodscited by 58
  16. 2021Differentially Private Query Release Through Adaptive Projectioncited by 50
  17. 2022Achieving Fairness at No Utility Cost via Data Reweighing with Influencecited by 38
  18. 2023TabMT: Generating tabular data with masked transformerscited by 36
  19. 2021Copula Flows for Synthetic Data Generationcited by 35
  20. 2022Private Synthetic Data for Multitask Learning and Marginal Queriescited by 35
  21. 2023FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methodscited by 33
  22. 2023Advancing Differential Privacy: Where We Are Now and Future Directions for Real-World Deploymentcited by 26
  23. 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
  24. 2024Elephants Never Forget: Testing Language Models for Memorization of Tabular Datacited by 19
  25. 2023Generating Private Synthetic Data with Genetic Algorithmscited by 18
  26. 2023DP-TBART: A Transformer-based Autoregressive Model for Differentially Private Tabular Data Generationcited by 13
  27. 2023Statistical Knowledge Assessment for Large Language Modelscited by 5
  28. 2023Privately generating tabular data using language modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 5.

16 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey of Synthetic Tabular Data Generationcited by 27
  2. 2024Private prediction for large-scale synthetic text generationcited by 24
  3. 2025A Case Study Exploring the Current Landscape of Synthetic Medical Record Generation with Commercial LLMscited by 13
  4. 2025Struct-Bench: A Benchmark for Differentially Private Structured Text Generationcited by 7
  5. 2024DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generatorscited by 5
  6. 2025Is API Access to LLMs Useful for Generating Private Synthetic Tabular Data?cited by 5
  7. 2025Clustering and Median Aggregation Improve Differentially Private Inferencecited by 5
  8. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 5
  9. 2024A text-to-tabular approach to generate synthetic patient data using LLMscited by 4
  10. 2025SynLLM: A Comparative Analysis of Large Language Models for Medical Tabular Synthetic Data Generation via Prompt Engineeringcited by 2
  11. 2026Differentially Private Synthetic Data via APIs 4: Tabular Datacited by 2
  12. 2025Assessing Generative Models for Structured Datacited by 1
4 more
  1. 2026Privately Fine-Tuned LLMs Preserve Temporal Dynamics in Tabular Datacited by 1
  2. 2025Beyond One-Size-Fits-All: Neural Networks for Differentially Private Tabular Data Synthesiscited by 0
  3. 2026Generative AI and Federated Learning for Intrusion Detection Systems: A Surveycited by 0
  4. 2026PEG-Tab: Sampling-Time Record Repair and Release Control for Tabular Synthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Tabular Data Synthesis with Differential Privacy: A Survey
  2. 2026Creation begins with understanding: LLMs as strategy designers for privacy-preserving tabular data synthesis
  3. 2024HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protection
  4. 2024Systematic Assessment of Tabular Data Synthesis
  5. 2026Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation
  6. 2024SafeSynthDP: Leveraging Large Language Models for Privacy-Preserving Synthetic Data Generation Using Differential Privacy
  7. 2024DP-TLDM: Differentially Private Tabular Latent Diffusion Model
  8. 2024Federated Diffusion Modeling with Differential Privacy for Tabular Data Synthesis