mlsanityData to 14 Oct 2026

KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server

arXiv 2410.05725 ↗First posted 8 Oct 2024cs.CR cs.AIcited by 11

On the map: Knowledge distillation → Federated, robust and private distillation

Abstract (arXiv)

The success of large language models (LLMs) facilitate many parties to fine-tune LLMs on their own private data. However, this practice raises privacy concerns due to the memorization of LLMs. Existing solutions, such as utilizing synthetic data for substitution, struggle to simultaneously improve performance and preserve privacy. They either rely on a local model for generation, resulting in a performance decline, or take advantage of APIs, directly exposing the data to API servers. To address this issue, we propose KnowledgeSG, a novel client-server framework which enhances synthetic data quality and improves model performance while ensuring privacy. We achieve this by learning local knowledge from the private data with differential privacy (DP) and distilling professional knowledge from the server. Additionally, inspired by federated learning, we transmit models rather than data between the client and server to prevent privacy leakage. Extensive experiments in medical and financial domains demonstrate the effectiveness of KnowledgeSG. Our code is now publicly available at https://github.com/wwh0411/KnowledgeSG.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Federated, robust and private distillation, one of 12 directions in Knowledge distillation. Builds on 49 of its 77 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

49 of 77 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Differentially Private Knowledge Distillation via Synthetic Text Generation1 Mar

Other references · 48

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  10. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  11. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  12. 2016Deep Learning with Differential Privacycited by 2,728
36 more
  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  2. 2020Extracting Training Data from Large Language Modelscited by 2,189
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  6. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  7. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  8. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  9. 2023BloombergGPT: A Large Language Model for Financecited by 939
  10. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  11. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  12. 2019Federated Learning with Differential Privacy: Algorithms and Performance Analysiscited by 536
  13. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  14. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  15. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  16. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 408
  17. 2021Opacus: User-Friendly Differential Privacy Library in PyTorchcited by 383
  18. 2023ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledgecited by 325
  19. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
  20. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
  21. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  22. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 195
  23. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 138
  24. 2024OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learningcited by 128
  25. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  26. 2024Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Modelscited by 99
  27. 2023AlpaCare:Instruction-tuned Large Language Models for Medical Applicationcited by 90
  28. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  29. 2024Differentially Private Synthetic Data via Foundation Model APIs 2: Textcited by 76
  30. 2023Harnessing large-language models to generate private synthetic textcited by 65
  31. 2023LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Modelscited by 60
  32. 2022DP-Rewrite: Towards Reproducibility and Transparency in Differentially Private Text Rewritingcited by 28
  33. 2023DB-GPT: Empowering Database Interactions with Private Large Language Modelscited by 27
  34. 2021Divergence Frontiers for Generative Models: Sample Complexity, Quantization Effects, and Frontier Integralscited by 18
  35. 2024Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classificationcited by 16
  36. 2020Really Useful Synthetic Data -- A Framework to Evaluate the Quality of Differentially Private Synthetic Datacited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 3.

11 papers.

  1. 2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 16
  2. 2025MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Toolscited by 11
  3. 2025MobileA3gent: Training Mobile GUI Agents Using Decentralized Self-Sourced Data from Diverse Userscited by 10
  4. 2025Clustering and Median Aggregation Improve Differentially Private Inferencecited by 5
  5. 2025RewardDS: Privacy-Preserving Fine-Tuning for Large Language Models via Reward Driven Data Synthesiscited by 3
  6. 2025FedMABench: Benchmarking Mobile Agents on Decentralized Heterogeneous User Datacited by 3
  7. 2026MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulationcited by 3
  8. 2025Cross-Document Cross-Lingual NLI via RST-Enhanced Graph Fusion and Interpretability Predictioncited by 2
  9. 2025LLMs Can Also Do Well! Breaking Barriers in Semantic Role Labeling via Large Language Modelscited by 1
  10. 2026Differentially Private Preference Data Synthesis for Large Language Model Alignmentcited by 1
  11. 2026PrivBench: A Holistic and Modular Benchmarking Platform for Evaluating Text-to-Text Privatizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains
  2. 2026Private Seeds, Public LLMs: Realistic and Privacy-Preserving Synthetic Data Generation
  3. 2025SynBench: A Benchmark for Differentially Private Text Generation
  4. 2026Learning to Refer: Client-Resolved Generation for Privacy-Aware Language Models
  5. 2024SafeSynthDP: Leveraging Large Language Models for Privacy-Preserving Synthetic Data Generation Using Differential Privacy
  6. 2024Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data
  7. 2025PPC-GPT: Federated Task-Specific Compression of Large Language Models via Pruning and Chain-of-Thought Distillation
  8. 2024Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directions