Mind the Hook: Source-Level Auditing of Privacy Defenses in Retrieval-Augmented Generation
On the map: Retrieval-augmented generation → Privacy and security of RAG
Abstract (arXiv)
Black-box privacy scores for retrieval-augmented generation (RAG) are difficult to interpret unless the audited defense's active pipeline hook is known. We propose an active-path audit: inventory source-level hooks over retrieval, retrieved content, and generation; map each metric to the leakage channel it observes; and validate generated-text effects with exact-match canaries. In our benchmark reimplementations, the DP-style defenses modify retrieval scores only: their generation hooks are TODO-flagged stubs that return responses unchanged. This active path explains why they affect membership-inference behavior but track No-Defense on generated-text named-entity leakage, measured by NEL_strict. By contrast, the end-to-end LPRAG path is canary-validated on the email channel, recovering 53/150 canaries under No-Defense and 0/150 under LPRAG. These findings concern our reimplementations on our stack, not released defenses or defense families; the contribution is a methodology and case study, not a universal ranking
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 15 of its 16 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
15 of 16 references.
Closest ancestors · 3
Same direction, newest first
- 2024Privacy-Preserving Retrieval-Augmented Generation with Differential Privacy6 Dec
- 2024Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applications21 Nov
- 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)23 Feb
Other references · 12
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2017Billion-scale similarity search with GPUscited by 2,497
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
- 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
- 2021Membership Inference Attacks From First Principlescited by 878
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
- 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 384
- 2026SafetyRepro: Configuration-Conditional Rank Instability on Alignment Benchmarkscited by 15
- 2026Pre-Registering the Detectable Effect: A Paired-MDE Budget for 4-bit Quantization Benchmarks, with a Pilot Auditcited by 8
- 2026Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canariescited by 7
- 2026Auditing and Fixing Economic Validity in Tabular Foundation Models for Discrete Choicecited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Security and Privacy in Retrieval-Augmented Generation: Architectures, Threats, Defenses, and Future Directions for Building Trustworthy Systems
- 2025Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generation
- 2026SD-RAG: A Prompt-Injection-Resilient Framework for Selective Disclosure in Retrieval-Augmented Generation
- 2026Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation
- 2024Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generation
- 2026Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directions
- 2026Is External Database Protection Static in Retrieval-Augmented Generation? Rethinking Privacy Preservation under Dynamic Queries
- 2026Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation