mlsanityData to 7 Oct 2026

Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation

arXiv 2604.07486 ↗First posted 8 Apr 2026cs.CR cs.AIcited by 2

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

Large language models (LLMs) have emerged as a powerful tool for synthetic data generation. A particularly important use case is producing synthetic replicas of private text, which requires carefully balancing privacy and utility. We propose Realistic and Privacy-Preserving Synthetic Data Generation (RPSG), which uses private seeds and integrates privacy-preserving strategies, including a formal differential privacy (DP) mechanism in the candidate selection, to generate realistic synthetic data. Comprehensive experiments against state-of-the-art private synthetic data generation methods demonstrate that RPSG achieves high fidelity to private data while providing strong privacy protection.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 38 of its 59 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

38 of 59 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data12 Sep
  2. 2024A Synthetic Dataset for Personal Attribute Inference11 Jun
  3. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Text4 Mar

Other references · 35

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  7. 2020Extracting Training Data from Large Language Modelscited by 2,144
  8. 2021Membership Inference Attacks From First Principlescited by 839
  9. 2017Renyi Differential Privacycited by 643
  10. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  11. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  12. 2021Differentially Private Fine-tuning of Language Modelscited by 392
23 more
  1. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 347
  2. 2023Fine-Tuning Language Models with Just Forward Passescited by 337
  3. 2018Scalable Private Learning with PATEcited by 266
  4. 2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
  5. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 240
  6. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 221
  7. 2023Does Synthetic Data Generation of LLMs Help Clinical Text Mining?cited by 151
  8. 2024LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMscited by 142
  9. 2019Well-Read Students Learn Better: On the Importance of Pre-training Compact Modelscited by 118
  10. 2023Machine Learning for Synthetic Data Generation: A Reviewcited by 118
  11. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  12. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  13. 2023Harnessing large-language models to generate private synthetic textcited by 65
  14. 2019Jointly Measuring Diversity and Quality in Text Generation Modelscited by 61
  15. 2023Reducing Privacy Risks in Online Self-Disclosures with Language Modelscited by 47
  16. 2022Memorization in NLP Fine-tuning Methodscited by 43
  17. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  18. 2024Privacy Preserving Prompt Engineering: A Surveycited by 24
  19. 2024IncogniText: Privacy-enhancing Conditional Text Anonymization via LLM-based Private Attribute Randomizationcited by 21
  20. 2024MIND: Math Informed syNthetic Dialogues for Pretraining LLMscited by 14
  21. 2024Montessori-Instruct: Generate Influential Training Data Tailored for Student Learningcited by 12
  22. 2024Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Datacited by 9
  23. 2024Masked Clinical Modelling: A Framework for Synthetic and Augmented Survival Data Generationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Aggregation-Aware Synthetic Text Generation Against Authorship Re-Identificationcited by 0
  2. 2026The Privacy-Hallucination Tradeoff in Differentially Private Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SafeSynthDP: Leveraging Large Language Models for Privacy-Preserving Synthetic Data Generation Using Differential Privacy
  2. 2024Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains
  3. 2024Differentially Private Tabular Data Synthesis using Large Language Models
  4. 2024Private Synthetic Text Generation with Diffusion Models
  5. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs
  6. 2025Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)
  7. 2026DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning
  8. 2026EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors