mlsanityData to 7 Oct 2026

DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning

arXiv 2602.18633 ↗First posted 20 Feb 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Differentially private (DP) synthetic data generation plays a pivotal role in developing large language models (LLMs) on private data, where data owners cannot provide eyes-on access to individual examples. Generating DP synthetic data typically involves a difficult trade-off. On one hand, DP finetuning methods train an LLM as a synthetic data generator with formal privacy guarantees, yet it still requires the raw content of private examples for model training. However, methods that avoid direct exposure to private data are bounded by an off-the-shelf, un-finetuned model, whose outputs often lack domain fidelity. Can we train an LLM to generate high-quality synthetic text without eyes-on access to individual private examples? In this work, we introduce Differentially Private Reinforcement Fine-Tuning (DP-RFT), an online reinforcement learning algorithm for synthetic data generation with LLMs. DP-RFT leverages DP-protected nearest-neighbor votes from an eyes-off private corpus as a reward signal for on-policy synthetic samples generated by an LLM. The LLM iteratively learns to generate synthetic data to maximize the expected DP votes through Proximal Policy Optimization (PPO). We evaluate DP-RFT for long-form and domain-specific synthetic data generation, such as news articles, meeting transcripts, and medical article abstracts. Our experiments show that DP-RFT closes the gap between private evolution and DP finetuning methods in terms of the fidelity and downstream utility of the generated synthetic data, while respecting the private data boundary.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 49 of its 59 references that are in the map. First posted on arXiv 20 Feb 2026.

Builds on

49 of 59 references.

References · 49

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  8. 2024Qwen2.5 Technical Reportcited by 4,713
  9. 2024GPT-4o System Cardcited by 4,467
  10. 2016Deep Learning with Differential Privacycited by 2,682
  11. 2020Extracting Training Data from Large Language Modelscited by 2,144
  12. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
37 more
  1. 2016Concrete Problems in AI Safetycited by 1,582
  2. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  3. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  4. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  5. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  6. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  7. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  8. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  9. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 363
  10. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 312
  11. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 246
  12. 2018Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoisingcited by 216
  13. 2024Stealing Part of a Production Language Modelcited by 152
  14. 2023How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacycited by 150
  15. 2023Emergent and Predictable Memorization in Large Language Modelscited by 147
  16. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  17. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 94
  18. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 80
  19. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Textcited by 74
  20. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  21. 2023Harnessing large-language models to generate private synthetic textcited by 65
  22. 2023Differentially Private Synthetic Data via Foundation Model APIs 1: Imagescited by 63
  23. 2023Privacy-Preserving In-Context Learning for Large Language Modelscited by 50
  24. 2024CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generationcited by 35
  25. 2022Pre-trained Perceptual Features Improve Differentially Private Image Generationcited by 28
  26. 2025Privacy Auditing of Large Language Modelscited by 22
  27. 2025A Survey on Privacy Risks and Protection in Large Language Modelscited by 20
  28. 2025Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidencecited by 16
  29. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 15
  30. 2025POPri: Private Federated Learning using Preference-Optimized Synthetic Datacited by 14
  31. 2024Dwell in the Beginning: How Language Models Embed Long Documents for Dense Retrievalcited by 13
  32. 2025Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Modelcited by 13
  33. 2025Synthetic Text Generation for Training Large Language Models via Gradient Matchingcited by 9
  34. 2025Struct-Bench: A Benchmark for Differentially Private Structured Text Generationcited by 7
  35. 2024Differentially Private Learning Needs Better Model Initialization and Self-Distillationcited by 3
  36. 2025RewardDS: Privacy-Preserving Fine-Tuning for Large Language Models via Reward Driven Data Synthesiscited by 3
  37. 2025Differentially Private In-context Learning via Sampling Few-shot Mixed with Zero-shot Outputscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Synthetic Data for any Differentiable Target
  2. 2024Privacy-Preserving Instructions for Aligning Large Language Models
  3. 2026Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation
  4. 2024SafeSynthDP: Leveraging Large Language Models for Privacy-Preserving Synthetic Data Generation Using Differential Privacy
  5. 2025ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Control
  6. 2024Differentially Private Tabular Data Synthesis using Large Language Models
  7. 2024Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains
  8. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data