Data to 14 Oct 2026

Privacy-Preserving In-Context Learning for Large Language Models

arXiv 2305.01639 ↗First posted 2 May 2023cs.LG cs.AI cs.CRcited by 50

Not on the map. Why?

Abstract (arXiv)

In-context learning (ICL) is an important capability of Large Language Models (LLMs), enabling these models to dynamically adapt based on specific, in-context exemplars, thereby improving accuracy and relevance. However, LLM's responses may leak the sensitive private information contained in in-context exemplars. To address this challenge, we propose Differentially Private In-context Learning (DP-ICL), a general paradigm for privatizing ICL tasks. The key idea for DP-ICL paradigm is generating differentially private responses through a noisy consensus among an ensemble of LLM's responses based on disjoint exemplar sets. Based on the general paradigm of DP-ICL, we instantiate several techniques showing how to privatize ICL for text classification and language generation. We evaluate DP-ICL on four text classification benchmarks and two language generation tasks, and our empirical results show that DP-ICL achieves a strong utility-privacy tradeoff.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Builds on 26 of its 54 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

26 of 54 references.

References · 26

2020Language Models are Few-Shot Learnerscited by 32,419
2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
2016Deep Learning with Differential Privacycited by 2,728
2015Character-level Convolutional Networks for Text Classificationcited by 2,330
2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
2017Renyi Differential Privacycited by 650
2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
2021Differentially Private Fine-tuning of Language Modelscited by 400
2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 346

14 more

2018Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergencescited by 242
2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
2022Active Example Selection for In-Context Learningcited by 196
2021Numerical Composition of Differential Privacycited by 180
2016Concentrated Differential Privacycited by 144
2021Hyperparameter Tuning with Renyi Differential Privacycited by 104
2022Differentially Private Optimization on Large Model at Small Costcited by 65
2022Exploring the Limits of Differentially Private Deep Learning with Group-wise Clippingcited by 52
2022Differentially Private Bias-Term Fine-tuning of Foundation Modelscited by 48
2019Practical Differentially Private Top-k Selection with Pay-what-you-get Compositioncited by 39
2021Computing Differential Privacy Guarantees for Heterogeneous Compositions Using FFTcited by 20
2022A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimizationcited by 11
2022A Joint Exponential Mechanism For Differentially Private Top-kcited by 10
2022Adaptive Private-K-Selection with Adaptive K and Application to Multi-label PATEcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 16; 2025: 16; 2026: 14.

50 papers.

Sort:

By area:

2023Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generationcited by 96
2023DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineercited by 57
2024Private Fine-tuning of Large Language Models with Zeroth-order Optimizationcited by 41
2024Privacy-Preserving Retrieval-Augmented Generation with Differential Privacycited by 39
2024Privacy-Preserving Instructions for Aligning Large Language Modelscited by 35
2023Identifying and Mitigating Privacy Risks Stemming from Language Models: A Surveycited by 30
2024PrE-Text: Training Language Models on Private Federated Data in the Age of LLMscited by 29
2024Privacy Preserving Prompt Engineering: A Surveycited by 25
2024Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuningcited by 24
2024Private prediction for large-scale synthetic text generationcited by 24
2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
2025Privacy Auditing of Large Language Modelscited by 23

38 more

2025The Canary's Echo: Auditing Privacy Risks of LLM-Generated Synthetic Textcited by 18
2025Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMscited by 16
2025Differentially Private Steering for Large Language Model Alignmentcited by 15
2024Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternativescited by 12
2024DP-TabICL: In-Context Learning with Differentially Private Tabular Datacited by 11
2024Data-adaptive Differentially Private Prompt Synthesis for In-Context Learningcited by 10
2025InvisibleInk: High-Utility and Low-Cost Text Generation with Differential Privacycited by 10
2024Can LLMs get help from other LLMs without revealing private information?cited by 9
2025Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generationcited by 8
2025AI Agent Behavioral Sciencecited by 6
2025Clustering and Median Aggregation Improve Differentially Private Inferencecited by 5
2024Adaptively Private Next-Token Prediction of Large Language Modelscited by 4
2025ContextLeak: Auditing Leakage in Private In-Context Learning Methodscited by 4
2026Benchmarking Empirical Privacy Protection for Adaptations of Large Language Modelscited by 4
2023Hot PATE: Private Aggregation of Distributions for Diverse Taskcited by 3
2025ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Controlcited by 3
2025Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Privatecited by 3
2024Privacy Profiles for Private Selectioncited by 2
2024NeurIPS 2023 Competition: Privacy Preserving Federated Learning Document VQAcited by 2
2025Privacy Preserving In-Context-Learning Framework for Large Language Modelscited by 2
2026EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectorscited by 2
2024Confidential Prompting: Privacy-preserving LLM Inference on Cloudcited by 1
2025Efficient and Privacy-Preserving Soft Prompt Transfer for LLMscited by 1
2025Sanitize Your Responses: Mitigating Privacy Leakage in Large Language Modelscited by 1
2025Differentially Private In-Context Learning with Nearest Neighbor Searchcited by 1
2026Leveraging Soft Prompts for Privacy Attacks in Federated Prompt Tuningcited by 1
2026NOIR: Privacy-Preserving Generation of Code with Open-Source LLMscited by 1
2026Differentially Private Multimodal In-Context Learningcited by 1
2026RedacBench: Can AI Erase Your Secrets?cited by 1
2026Differentially Private Preference Data Synthesis for Large Language Model Alignmentcited by 1
2026Privacy Vulnerabilities of Attention Layers in Tabular Foundation Models and Protection of High-Risk Queriescited by 1
2025Transforming Tuberculosis Care: Optimizing Large Language Models For Enhanced Clinician-Patient Communicationcited by 0
2026Private PoEtry: Private In-Context Learning via Product of Expertscited by 0
2026DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuningcited by 0
2026Membership Inference Attacks for Retrieval Based In-Context Learning for Document Question Answeringcited by 0
2026Differentially Private Datastore Generation for Retrieval-Augmented Inferencecited by 0
2026TabPATE: Differentially Private Tabular In-Context Learning Without Public Datacited by 0
2026Federated Agent Optimizationcited by 0