# VortexPIA: Indirect Prompt Injection Attack against LLMs for Efficient Extraction of User Privacy

arXiv: https://arxiv.org/abs/2510.04261 · HTML: https://mlsanity.com/paper/2510.04261/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 5 Oct 2025.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 27 of 27)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024)
- [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024)
- [ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs](https://mlsanity.com/paper/2309.13007/) (2023)
- [Analyzing Leakage of Personally Identifiable Information in Language Models](https://mlsanity.com/paper/2302.00539/) (2023)
- [Optimization-based Prompt Injection Attack to LLM-as-a-Judge](https://mlsanity.com/paper/2403.17710/) (2024)
- [Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents](https://mlsanity.com/paper/2503.00061/) (2025)
- [Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification](https://mlsanity.com/paper/2407.20859/) (2024)
- [Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench](https://mlsanity.com/paper/2410.22108/) (2024)
- [PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles](https://mlsanity.com/paper/2410.17127/) (2024)
- [Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild](https://mlsanity.com/paper/2407.11438/) (2024)
- [Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2406.07973/) (2024)
- [Searching for Privacy Risks in LLM Agents via Simulation](https://mlsanity.com/paper/2508.10880/) (2025)
- [Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness](https://mlsanity.com/paper/2505.22960/) (2025)
- [Fun-tuning: Characterizing the Vulnerability of Proprietary LLMs to Optimization-based Prompt Injection Attacks via the Fine-Tuning Interface](https://mlsanity.com/paper/2501.09798/) (2025)
- [Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Training](https://mlsanity.com/paper/2502.19726/) (2025)
- [Defense against Prompt Injection Attacks via Mixture of Encodings](https://mlsanity.com/paper/2504.07467/) (2025)
- [Malicious LLM-Based Conversational AI Makes Users Reveal Personal Information](https://mlsanity.com/paper/2506.11680/) (2025)
- [Exploring User Security and Privacy Attitudes and Concerns Toward the Use of General-Purpose LLM Chatbots for Mental Health](https://mlsanity.com/paper/2507.10695/) (2025)
- [Mitigating Memorization In Language Models](https://mlsanity.com/paper/2410.02159/) (2024)
- [PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization](https://mlsanity.com/paper/2505.09921/) (2025)
- [Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning](https://mlsanity.com/paper/2509.13755/) (2025)
- [Be Cautious When Merging Unfamiliar LLMs: A Phishing Model Capable of Stealing Privacy](https://mlsanity.com/paper/2502.11533/) (2025)

## Built on by (most cited first, 7 of 7)

- [The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis](https://mlsanity.com/paper/2602.10453/) (2026)
- [Taxonomy, Evaluation and Exploitation of IPI-Centric LLM Agent Defense Frameworks](https://mlsanity.com/paper/2511.15203/) (2025)
- [ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior](https://mlsanity.com/paper/2512.05745/) (2025)
- [MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agents](https://mlsanity.com/paper/2605.09530/) (2026)
- [AgentRaft: Automated Detection of Data Over-Exposure in LLM Agents](https://mlsanity.com/paper/2603.07557/) (2026)
- [Indirect Prompt Injection in the Wild: An Empirical Study of Prevalence, Techniques, and Objectives](https://mlsanity.com/paper/2604.27202/) (2026)
- [Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control](https://mlsanity.com/paper/2609.14003/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have been widely deployed in Conversational AIs (CAIs), while exposing privacy and security threats. Recent research shows that LLM-based CAIs can be manipulated to extract private information from human users, posing serious security threats. However, the methods proposed in that study rely on a white-box setting that adversaries can directly modify the system prompt. This condition is unlikely to hold in real-world deployments. The limitation raises a critical question: can unprivileged attackers still induce such privacy risks in practical LLM-integrated applications? To address this question, we propose textscVortexPIA, a novel indirect prompt injection attack that induces privacy extraction in LLM-integrated applications under black-box settings. By injecting token-efficient data containing false memories, textscVortexPIA misleads LLMs to actively request private information in batches. Unlike prior methods, textscVortexPIA allows attackers to flexibly define multiple categories of sensitive data. We evaluate textscVortexPIA on six LLMs, covering both traditional and reasoning LLMs, across four benchmark datasets. The results show that textscVortexPIA significantly outperforms baselines and achieves state-of-the-art (SOTA) performance. It also demonstrates efficient privacy requests, reduced token consumption, and enhanced robustness against defense mechanisms. We further validate textscVortexPIA on multiple realistic open-source LLM-integrated applications, demonstrating its practical effectiveness.