mlsanityData to 14 Oct 2026

Data-adaptive Differentially Private Prompt Synthesis for In-Context Learning

arXiv 2410.12085 ↗First posted 15 Oct 2024cs.CR cs.AI cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) rely on the contextual information embedded in examples/demonstrations to perform in-context learning (ICL). To mitigate the risk of LLMs potentially leaking private information contained in examples in the prompt, we introduce a novel data-adaptive differentially private algorithm called AdaDPSyn to generate synthetic examples from the private dataset and then use these synthetic examples to perform ICL. The objective of AdaDPSyn is to adaptively adjust the noise level in the data synthesis mechanism according to the inherent statistical properties of the data, thereby preserving high ICL accuracy while maintaining formal differential privacy guarantees. A key innovation in AdaDPSyn is the Precision-Focused Iterative Radius Reduction technique, which dynamically refines the aggregation radius - the scope of data grouping for noise addition - based on patterns observed in data clustering, thereby minimizing the amount of additive noise. We conduct extensive experiments on standard benchmarks and compare AdaDPSyn with DP few-shot generation algorithm (Tang et al., 2023). The experiments demonstrate that AdaDPSyn not only outperforms DP few-shot generation, but also maintains high accuracy levels close to those of non-private baselines, providing an effective solution for ICL with privacy protection.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 51 of its 72 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

51 of 72 references.

References · 51

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  4. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  6. 2017Renyi Differential Privacycited by 650
  7. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  8. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  9. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  10. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  11. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
  12. 2018Scalable Private Learning with PATEcited by 269
38 more
  1. 2018Subsampled R'enyi Differential Privacy and Analytical Moments Accountantcited by 236
  2. 2019Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbationscited by 119
  3. 2020BERT-of-Theseus: Compressing BERT by Progressive Module Replacingcited by 113
  4. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  5. 2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 96
  6. 2019Hypothesis Testing Interpretations and Renyi Differential Privacycited by 85
  7. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 83
  8. 2022Differentially Private Language Models for Secure Data Sharingcited by 68
  9. 2021Differential privacy and robust statistics in high dimensionscited by 64
  10. 2023Locally Differentially Private Document Generation Using Zero Shot Promptingcited by 63
  11. 2018Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domaincited by 61
  12. 2023DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineercited by 57
  13. 2019Average-Case Averages: Private Algorithms for Smooth Sensitivity and Mean Estimationcited by 52
  14. 2022A Customized Text Sanitization Mechanism with Differential Privacycited by 52
  15. 2022Exploring the Limits of Differentially Private Deep Learning with Group-wise Clippingcited by 52
  16. 2023Privacy-Preserving In-Context Learning for Large Language Modelscited by 50
  17. 2024Private Fine-tuning of Large Language Models with Zeroth-order Optimizationcited by 41
  18. 2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 35
  19. 2020Differentially Private Clustering: Tight Approximation Ratioscited by 34
  20. 2022Differentially Private Decoding in Large Language Modelscited by 34
  21. 2022Submix: Practical Private Prediction for Large-Scale Language Modelscited by 31
  22. 2016Locating a Small Cluster Privatelycited by 20
  23. 2017Clustering Algorithms for the Centralized and Local Modelscited by 18
  24. 2023Are Chatbots Ready for Privacy-Sensitive Applications? An Investigation into Input Regurgitation and Prompt-Induced Sanitizationcited by 17
  25. 2024Prompt Public Large Language Models to Synthesize Data for Private On-device Applicationscited by 17
  26. 2016Differentially Private Random Decision Forests using Smooth Sensitivitycited by 16
  27. 2023Guiding Text-to-Text Privatization by Syntaxcited by 14
  28. 2023Driving Context into Text-to-Text Privatizationcited by 13
  29. 2022Privacy-Preserving Domain Adaptation of Semantic Parserscited by 12
  30. 2024DP-TabICL: In-Context Learning with Differentially Private Tabular Datacited by 11
  31. 2023In-context Learning as Maintaining Coherency: A Study of On-the-fly Machine Translation Using Large Language Modelscited by 10
  32. 2022Adaptive Private-K-Selection with Adaptive K and Application to Multi-label PATEcited by 9
  33. 2022Can Querying for Bias Leak Protected Attributes? Achieving Privacy With Smooth Sensitivitycited by 8
  34. 2022Generalized PTR: User-Friendly Recipes for Data-Adaptive Algorithms with Differential Privacycited by 6
  35. 2024Privacy-preserving Fine-tuning of Large Language Models through Flatnesscited by 6
  36. 2018An end-to-end Differentially Private Latent Dirichlet Allocation Using a Spectral Algorithmcited by 4
  37. 2020Differentially Private Deep Learning with Smooth Sensitivitycited by 4
  38. 2022Renyi Differential Privacy of Propose-Test-Release and Applications to Private and Robust Machine Learningcited by 4

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.

  1. 2025Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)cited by 6
  2. 2025Clustering and Median Aggregation Improve Differentially Private Inferencecited by 5
  3. 2026On the Price of Privacy for Language Identification and Generationcited by 3
  4. 2025DP-GTR: Differentially Private Prompt Protection via Group Text Rewritingcited by 2
  5. 2025Anti-adversarial Learning: Desensitizing Prompts for Large Language Modelscited by 2
  6. 2026EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectorscited by 2
  7. 2025Differentially Private In-Context Learning with Nearest Neighbor Searchcited by 1
  8. 2026Differentially Private Preference Data Synthesis for Large Language Model Alignmentcited by 1
  9. 2026Membership Inference Attacks for Retrieval Based In-Context Learning for Document Question Answeringcited by 0
  10. 2026FinancialAuditBench: Benchmark Construction under Differential Privacy Using Real-World Priorscited by 0