# Walking the Embedding Space: Datastore Extraction from Multimodal RAG

arXiv: https://arxiv.org/abs/2610.01871 · HTML: https://mlsanity.com/paper/2610.01871/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 26 of its 55 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)
- Retrieval-augmented generation / [Privacy and security of RAG](https://mlsanity.com/rag/privacy-and-security-of-rag/)
- LLM safety and security / [Vision-language and text-to-image safety](https://mlsanity.com/safety/vision-language-and-text-to-image-safety/)

## Builds on (most cited first, 26 of 26)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023)
- [Visual Adversarial Examples Jailbreak Aligned Large Language Models](https://mlsanity.com/paper/2306.13213/) (2023)
- [Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models](https://mlsanity.com/paper/2307.14539/) (2023)
- [Using an LLM to Help With Code Understanding](https://mlsanity.com/paper/2307.08177/) (2023)
- [Image Hijacks: Adversarial Images can Control Generative Models at Runtime](https://mlsanity.com/paper/2309.00236/) (2023)
- [Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining](https://mlsanity.com/paper/2408.02657/) (2024)
- [MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models](https://mlsanity.com/paper/2410.13085/) (2024)
- [Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2402.17840/) (2024)
- [A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languages](https://mlsanity.com/paper/2410.03981/) (2024)
- [INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent](https://mlsanity.com/paper/2412.18174/) (2024)
- [Assessing Prompt Injection Risks in 200+ Custom GPTs](https://mlsanity.com/paper/2311.11538/) (2023)
- [Making LLMs Worth Every Penny: Resource-Limited Text Classification in Banking](https://mlsanity.com/paper/2311.06102/) (2023)
- [Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreaking](https://mlsanity.com/paper/2409.08045/) (2024)
- [Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.00306/) (2025)
- [REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language Models](https://mlsanity.com/paper/2402.07016/) (2024)
- [Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases](https://mlsanity.com/paper/2412.18295/) (2024)
- [AutoLAW: Augmented Legal Reasoning through Legal Precedent Prediction](https://mlsanity.com/paper/2106.16034/) (2021)
- [Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generation](https://mlsanity.com/paper/2503.06254/) (2025)
- [Enhanced Multimodal RAG-LLM for Accurate Visual Question Answering](https://mlsanity.com/paper/2412.20927/) (2024)
- [Beyond Text: Unveiling Privacy Vulnerabilities in Multi-modal Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.13957/) (2025)
- [HV-Attack: Hierarchical Visual Attack for Multimodal Retrieval Augmented Generation](https://mlsanity.com/paper/2511.15435/) (2025)
- [Do Multimodal RAG Systems Leak Data? A Comprehensive Evaluation of Membership Inference and Image Caption Retrieval Attacks](https://mlsanity.com/paper/2601.17644/) (2026)

## Abstract (arXiv)

Multimodal Retrieval-Augmented Generation (MRAG) has emerged as a reliable and cost-effective technique of grounding the generative capabilities of Multimodal Large Language Models (MLLMs) into relevant, up-to-date, external knowledge. Despite presenting several benefits, such as reducing hallucinatory behavior, they also introduce new attack surfaces, including leakage of private information and vulnerabilities against data extraction attacks. In this paper, we introduce immrag, an adaptive and automatic data extraction attack procedure operating in a black box setting against image-returning MRAG, a configuration in which the retrieved visual artifact is itself the response. Each query blends an attacker-held shadow image with an image already recovered from the system, and relevance-weighted resampling steers subsequent queries towards regions of the embedding space that still yield novel retrievals. Unlike current extraction attacks that aim to persuade the model towards data leakage by placing a malicious query as a textual prompt, immrag embeds the malicious instructions inside a user-given input image. We evaluate immrag on three plausible and distinct real-world scenarios: medical assistant, document-focused helper and general purpose tool. The experiments involve the study of the effectiveness of the attack on multiple CLIP-family retrievers, as well as the impact of various generators. A single 2500-query run reconstructs up to 611 distinct radiology images, 566 document scans and 416 general-purpose images under local-feature correspondence, and reaches up to 5.6× as many distinct datastore items as a non-adaptive baseline. Our results show the urgent need for safeguards specifically designed for multimodal data.