mlsanityData to 7 Oct 2026

RewardDS: Privacy-Preserving Fine-Tuning for Large Language Models via Reward Driven Data Synthesis

arXiv 2502.18517 ↗First posted 23 Feb 2025cs.CR cs.AIcited by 3

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

The success of large language models (LLMs) has attracted many individuals to fine-tune them for domain-specific tasks by uploading their data. However, in sensitive areas like healthcare and finance, privacy concerns often arise. One promising solution is to generate synthetic data with Differential Privacy (DP) guarantees to replace private data. However, these synthetic data contain significant flawed data, which are considered as noise. Existing solutions typically rely on naive filtering by comparing ROUGE-L scores or embedding similarities, which are ineffective in addressing the noise. To address this issue, we propose textitRewardDS, a novel privacy-preserving framework that fine-tunes a reward proxy model and uses reward signals to guide the synthetic data generation. Our textitRewardDS introduces two key modules, Reward Guided Filtering and Self-Optimizing Refinement, to both filter and refine the synthetic data, effectively mitigating the noise. Extensive experiments across medical, financial, and code generation domains demonstrate the effectiveness of our method.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 36 of its 55 references that are in the map. First posted on arXiv 23 Feb 2025.

Builds on

36 of 55 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Contrastive Private Data Synthesis via Weighted Multi-PLM Fusion1 Feb
  2. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Text4 Mar

Other references · 34

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2016Deep Learning with Differential Privacycited by 2,682
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  10. 2020Extracting Training Data from Large Language Modelscited by 2,144
  11. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  12. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
22 more
  1. 2023BloombergGPT: A Large Language Model for Financecited by 923
  2. 2021Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Modelscited by 462
  3. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  4. 2021Differentially Private Fine-tuning of Language Modelscited by 392
  5. 2020Label-Only Membership Inference Attackscited by 371
  6. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 324
  7. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 322
  8. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  9. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  10. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  11. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  12. 2023Harnessing large-language models to generate private synthetic textcited by 65
  13. 2023Differentially Private Synthetic Data via Foundation Model APIs 1: Imagescited by 63
  14. 2024Can Perplexity Reflect Large Language Model's Ability in Long Text Understanding?cited by 45
  15. 2023PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Modelscited by 35
  16. 2017Learning Differentially Private Recurrent Language Modelscited by 32
  17. 2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
  18. 2019Glyph: Fast and Accurately Training Deep Neural Networks on Encrypted Datacited by 28
  19. 2024PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restorationcited by 27
  20. 2024Large Language Models are Advanced Anonymizerscited by 22
  21. 2024Differentially Private Knowledge Distillation via Synthetic Text Generationcited by 18
  22. 2024KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Servercited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuningcited by 0
  2. 2026Concordia: Self-Improving Synthetic Tables for Federated LLMscited by 0
  3. 2026LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communicationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SafeSynthDP: Leveraging Large Language Models for Privacy-Preserving Synthetic Data Generation Using Differential Privacy
  2. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data
  3. 2024Privacy-Preserving Instructions for Aligning Large Language Models
  4. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs
  5. 2026Privacy-Preserving Reinforcement Learning from Human Feedback via Decoupled Reward Modeling
  6. 2024Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains
  7. 2024Differentially Private Tabular Data Synthesis using Large Language Models
  8. 2026Phantoms and Disclosures: A Statistical Framework for Auditing Privacy in Synthetic Data