mlsanityData to 14 Oct 2026

Privacy Implications of Retrieval-Based Language Models

arXiv 2305.14888 ↗First posted 24 May 2023cs.CL cs.CR cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Retrieval-based language models (LMs) have demonstrated improved interpretability, factuality, and adaptability compared to their parametric counterparts, by incorporating retrieved text from external datastores. While it is well known that parametric models are prone to leaking private data, it remains unclear how the addition of a retrieval datastore impacts model privacy. In this work, we present the first study of privacy risks in retrieval-based LMs, particularly kNN-LMs. Our goal is to explore the optimal design and training procedure in domains where privacy is of concern, aiming to strike a balance between utility and privacy. Crucially, we find that kNN-LMs are more susceptible to leaking private information from their private datastore than parametric models. We further explore mitigations of privacy risks. When privacy information is targeted and readily detected in the text, we find that a simple sanitization step would completely eliminate the risks, while decoupling query and key encoders achieves an even better utility-privacy trade-off. Otherwise, we consider strategies of mixing public and private data in both datastore and encoder training. While these methods offer modest improvements, they leave considerable room for future work. Together, our findings provide insights for practitioners to better understand and mitigate privacy risks in retrieval-based LMs. Our code is available at: https://github.com/Princeton-SysML/kNNLMprivacy .

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

30 of 51 references.

References · 30

  1. 2016Pointer Sentinel Mixture Modelscited by 3,096
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  5. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  6. 2018Hierarchical Neural Story Generationcited by 1,193
  7. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  8. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  9. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  10. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  11. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  12. 2018Exploiting Unintended Feature Leakage in Collaborative Learningcited by 520
18 more
  1. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 412
  2. 2021Differentially Private Fine-tuning of Language Modelscited by 400
  3. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
  4. 2021Counterfactual Memorization in Neural Language Modelscited by 156
  5. 2022Training Language Models with Memory Augmentationcited by 113
  6. 2022Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipecited by 108
  7. 2019Gmail Smart Compose: Real-Time Assisted Writingcited by 98
  8. 2021Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?cited by 93
  9. 2021Efficient Nearest Neighbor Language Modelscited by 93
  10. 2022Recovering Private Text in Federated Learning of Language Modelscited by 70
  11. 2021TAG: Gradient Attack on Transformer-based Language Modelscited by 63
  12. 2021Selective Differential Privacy for Language Modelingcited by 62
  13. 2022LAMP: Extracting Text from Gradients with Language Model Priorscited by 54
  14. 2022Nonparametric Masked Language Modelingcited by 40
  15. 2020TextHide: Tackling Data Privacy in Language Understanding Taskscited by 33
  16. 2023Why do Nearest Neighbor Language Models Work?cited by 29
  17. 2023Advancing Differential Privacy: Where We Are Now and Future Directions for Real-World Deploymentcited by 27
  18. 2022Reasoning over Public and Private Data in Retrieval-Based Systemscited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 14; 2025: 8; 2026: 7.

32 papers.

Sort:

By area:

  1. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 261
  2. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 101
  3. 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 89
  4. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  5. 2024Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systemscited by 76
  6. 2023SILO Language Models: Isolating Legal Risk In a Nonparametric Datastorecited by 71
  7. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 51
  8. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 47
  9. 2024Privacy Preserving Prompt Engineering: A Surveycited by 25
  10. 2024Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuningcited by 24
  11. 2024AI Risk Management Should Incorporate Both Safety and Securitycited by 17
  12. 2025A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakagecited by 16
20 more
  1. 2023On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlookcited by 15
  2. 2024Developing Story: Case Studies of Generative AI's Use in Journalismcited by 10
  3. 2025Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMscited by 10
  4. 2025Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generationcited by 8
  5. 2024Copyright-Protected Language Generation via Adaptive Model Fusioncited by 7
  6. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systemscited by 7
  7. 2025Automated Privacy Information Annotation in Large Language Model Interactionscited by 6
  8. 2024Strong Copyright Protection for Language Models via Adaptive Model Fusioncited by 4
  9. 2025SMA: Who Said That? Auditing Membership Leakage in Semi-Black-box RAG Controllingcited by 3
  10. 2024Initial Exploration of Zero-Shot Privacy Utility Tradeoffs in Tabular Data Using GPT-4cited by 2
  11. 2024Federated Domain-Specific Knowledge Transfer on Large Language Models Using Synthetic Datacited by 2
  12. 2025External Data Extraction Attacks against Retrieval-Augmented Large Language Modelscited by 2
  13. 2025Diverse And Private Synthetic Datasets Generation for RAG evaluation: A multi-agent frameworkcited by 1
  14. 2026Towards Privacy-Preserving Machine Translation at the Inference Stage: A New Task and Benchmarkcited by 1
  15. 2024A Survey on the Real Power of ChatGPTcited by 0
  16. 2026A Case Study on the Impact of Anonymization Along the RAG Pipelinecited by 0
  17. 2026PRAG: End-to-End Privacy-Preserving Retrieval-Augmented Generationcited by 0
  18. 2026PYTHALAB-MERA: Validation-Grounded Memory, Retrieval, and Acceptance Control for Frozen-LLM Coding Agentscited by 0
  19. 2026Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelitycited by 0
  20. 2026Is External Database Protection Static in Retrieval-Augmented Generation? Rethinking Privacy Preservation under Dynamic Queriescited by 0