mlsanityData to 14 Oct 2026

Do Multimodal RAG Systems Leak Data? A Comprehensive Evaluation of Membership Inference and Image Caption Retrieval Attacks

arXiv 2601.17644 ↗First posted 25 Jan 2026cs.CR cs.AIcited by 3

On the map: Retrieval-augmented generation → Privacy and security of RAG

also in 1 more
Abstract (arXiv)

The growing adoption of multimodal Retrieval-Augmented Generation (mRAG) pipelines for vision-centric tasks (e.g., visual QA) introduces important privacy challenges. In particular, while mRAG provides a practical capability to connect private datasets and improve model performance, it risks the leakage of private information from these datasets. In this paper, we perform an empirical study to analyze the privacy risks inherent in the mRAG pipeline observed through standard model prompting. Specifically, we implement a case study that attempts to determine whether a visual asset (e.g., image) is included in the mRAG, and, if present, to leak the metadata (e.g., caption) related to it. Our findings highlight the need for privacy-preserving mechanisms and motivate future research on mRAG privacy. Our code is published online: https://github.com/aliwister/mrag-attack-eval.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 21 of its 36 references that are in the map. First posted on arXiv 25 Jan 2026.

Builds on

21 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025MrM: Black-Box Membership Inference Attacks against Multimodal RAG Systems9 Jun
  2. 2025Beyond Text: Unveiling Privacy Vulnerabilities in Multi-modal Retrieval-Augmented Generation20 May
  3. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)23 Feb

Other references · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  3. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  4. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  5. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  6. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 220
  7. 2024An Introduction to Vision-Language Modelingcited by 178
  8. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 170
  9. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 138
  10. 2024Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehensioncited by 138
  11. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 125
  12. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Modelscited by 85
6 more
  1. 2024EchoSight: Advancing Visual-Language Models with Wiki Knowledgecited by 62
  2. 2024ROCOv2: Radiology Objects in COntext Version 2, an Updated Multimodal Image Datasetcited by 54
  3. 2024XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tagscited by 9
  4. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthinesscited by 7
  5. 2025VisRet: Visualization Improves Knowledge-Intensive Text-to-Image Retrievalcited by 3
  6. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026URAG: A Benchmark for Uncertainty Quantification in Retrieval-Augmented Large Language Modelscited by 3
  2. 2026Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directionscited by 2
  3. 2026Walking the Embedding Space: Datastore Extraction from Multimodal RAGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025HV-Attack: Hierarchical Visual Attack for Multimodal Retrieval Augmented Generation
  2. 2025Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generation
  3. 2026Hidden in the Metadata: Stealth Poisoning Attacks on Multimodal Retrieval-Augmented Generation
  4. 2026When Visual Privacy Protection Meets Multimodal Large Language Models
  5. 2025Assessing Visual Privacy Risks in Multimodal AI: A Novel Taxonomy-Grounded Evaluation of Vision-Language Models
  6. 2022Membership Inference Attacks Against Text-to-image Generation Models
  7. 2026Identity-Decoupled Anonymization for Visual Evidence in Multi-modal Retrieval-Augmented Generation
  8. 2026Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges