Data to 7 Oct 2026

Differentially Private Synthetic Data via Foundation Model APIs 2: Text

arXiv 2403.01749 ↗First posted 4 Mar 2024cs.CLcited by 74

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

Text data has become extremely valuable due to the emergence of machine learning algorithms that learn from it. A lot of high-quality text data generated in the real world is private and therefore cannot be shared or used freely due to privacy concerns. Generating synthetic replicas of private text data with a formal privacy guarantee, i.e., differential privacy (DP), offers a promising and scalable solution. However, existing methods necessitate DP finetuning of large language models (LLMs) on private data to generate DP synthetic data. This approach is not viable for proprietary LLMs (e.g., GPT-3.5) and also demands considerable computational resources for open-source LLMs. Lin et al. (2024) recently introduced the Private Evolution (PE) algorithm to generate DP synthetic images with only API access to diffusion models. In this work, we propose an augmented PE algorithm, named Aug-PE, that applies to the complex setting of text. We use API access to an LLM and generate DP synthetic text without any model training. We conduct comprehensive experiments on three benchmark datasets. Our results demonstrate that Aug-PE produces DP synthetic text that yields competitive utility with the SOTA DP finetuning baselines. This underscores the feasibility of relying solely on API access of LLMs to produce high-quality DP synthetic texts, thereby facilitating more accessible routes to privacy-preserving LLM applications. Our code and data are available at https://github.com/AI-secure/aug-pe.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 38 of its 58 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

38 of 58 references.

References · 38

2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
2023GPT-4 Technical Reportcited by 18,132
2022Training language models to follow instructions with human feedbackcited by 15,712
2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
2022Classifier-Free Diffusion Guidancecited by 5,717
2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
2016Deep Learning with Differential Privacycited by 2,682

26 more

2020Extracting Training Data from Large Language Modelscited by 2,144
2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
2021Membership Inference Attacks From First Principlescited by 839
2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
2021Differentially Private Fine-tuning of Language Modelscited by 392
2023Fine-Tuning Language Models with Just Forward Passescited by 337
2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
2023Membership Inference Attacks against Language Models via Neighbourhood Comparisoncited by 249
2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 247
2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 216
2021Large-Scale Differentially Private BERTcited by 117
2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 115
2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
2020Hiding Among the Clones: A Simple and Nearly Optimal Analysis of Privacy Amplification by Shufflingcited by 96
2021Large Scale Private Learning via Low-rank Reparametrizationcited by 87
2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 86
2022Differentially Private Language Models for Secure Data Sharingcited by 68
2023Harnessing large-language models to generate private synthetic textcited by 65
2022The Limits of Word Level Differential Privacycited by 63
2023Differentially Private Synthetic Data via Foundation Model APIs 1: Imagescited by 63
2023Locally Differentially Private Document Generation Using Zero Shot Promptingcited by 63
2022Exploring the Limits of Differentially Private Deep Learning with Group-wise Clippingcited by 52
2022Differentially Private Bias-Term Fine-tuning of Foundation Modelscited by 47
2021TEM: High Utility Metric Differential Privacy on Textcited by 43

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 35; 2026: 24.

74 papers.

Sort:

By area:

2024Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Datacited by 52
2024Is My Data in Your Retrieval Database? Membership Inference Attacks Against Retrieval Augmented Generationcited by 44
2024Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuningcited by 40
2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
2024Privacy Preserving Prompt Engineering: A Surveycited by 24
2024Private prediction for large-scale synthetic text generationcited by 24
2024Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Surveycited by 19
2024RAG with Differential Privacycited by 18
2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Textcited by 18
2024Prompt Public Large Language Models to Synthesize Data for Private On-device Applicationscited by 17
2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 15
2025POPri: Private Federated Learning using Preference-Optimized Synthetic Datacited by 14

38 more

2025Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Modelcited by 13
2024KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Servercited by 11
2025DPImageBench: A Unified Benchmark for Differentially Private Image Synthesiscited by 11
2024Synth-Empathy: Towards High-Quality Synthetic Empathy Datacited by 10
2025Synthesize Privacy-Preserving High-Resolution Images via Private Textual Intermediariescited by 8
2025Struct-Bench: A Benchmark for Differentially Private Structured Text Generationcited by 7
2024SynthVLM: Towards High-Quality and Efficient Synthesis of Image-Caption Datasets for Vision-Language Modelscited by 6
2025From Easy to Hard: Building a Shortcut for Differentially Private Image Synthesiscited by 6
2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
2025Is API Access to LLMs Useful for Generating Private Synthetic Tabular Data?cited by 5
2025Do Large Language Models (Really) Need Statistical Foundations?cited by 5
2025Scaling Textual Gradients via Sampling-Based Momentumcited by 5
2025Clustering and Median Aggregation Improve Differentially Private Inferencecited by 5
2025Evaluating Differentially Private Generation of Domain-Specific Textcited by 5
2025Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)cited by 5
2024Adaptively Private Next-Token Prediction of Large Language Modelscited by 4
2025Generating Synthetic Data with Formal Privacy Guarantees: State of the Art and the Road Aheadcited by 4
2025Towards Harnessing the Collaborative Power of Large and Small Models for Domain Taskscited by 4
2025A Survey of LLM × DATAcited by 4
2025SynBench: A Benchmark for Differentially Private Text Generationcited by 4
2024Differentially Private Learning Needs Better Model Initialization and Self-Distillationcited by 3
2025RewardDS: Privacy-Preserving Fine-Tuning for Large Language Models via Reward Driven Data Synthesiscited by 3
2025A Comprehensive Review on Harnessing Large Language Models to Overcome Recommender System Challengescited by 3
2026Privasis: Synthesizing the Largest "Public" Private Dataset from Scratchcited by 3
2024Differentially Private Kernel Density Estimationcited by 2
2025RAPID: Retrieval Augmented Training of Differentially Private Diffusion Modelscited by 2
2025Private Text Generation by Seeding Large Language Model Promptscited by 2
2025Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applicationscited by 2
2025Data Swarms: Optimizable Generation of Synthetic Evaluation Datacited by 2
2025Private Evolution Convergescited by 2
2025Privacy Preserving In-Context-Learning Framework for Large Language Modelscited by 2
2025ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Controlcited by 2
2025Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Privatecited by 2
2026FedMosaic: Federated Retrieval-Augmented Generation via Parametric Adapterscited by 2
2026EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectorscited by 2
2026Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generationcited by 2
2026Differentially Private Synthetic Data via APIs 4: Tabular Datacited by 2
2025Optimal Differentially Private Sampling of Unbounded Gaussianscited by 1

50 most cited of 74 All 74 (JSON)

Similar, not cited

Close in wording, no citation either way