# Prompt-in-Content Attacks: Exploiting Uploaded Inputs to Hijack LLM Behavior

arXiv: https://arxiv.org/abs/2508.19287 · HTML: https://mlsanity.com/paper/2508.19287/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 14 of 14)

- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023)
- [Defending Against Indirect Prompt Injection Attacks With Spotlighting](https://mlsanity.com/paper/2403.14720/) (2024)
- [Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models](https://mlsanity.com/paper/2312.14197/) (2023)
- [Security and Privacy Challenges of Large Language Models: A Survey](https://mlsanity.com/paper/2402.00888/) (2024)
- [Large Language Models in Law: A Survey](https://mlsanity.com/paper/2312.03718/) (2023)
- [Automatic and Universal Prompt Injection Attacks against Large Language Models](https://mlsanity.com/paper/2403.04957/) (2024)
- [A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems](https://mlsanity.com/paper/2402.18649/) (2024)
- [A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law](https://mlsanity.com/paper/2405.01769/) (2024)
- [Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2311.12351/) (2023)
- [SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models](https://mlsanity.com/paper/2305.19308/) (2023)
- [Goal-guided Generative Prompt Injection Attack on Large Language Models](https://mlsanity.com/paper/2404.07234/) (2024)
- [FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attacks](https://mlsanity.com/paper/2410.21492/) (2024)
- [Safeguarding Crowdsourcing Surveys from ChatGPT with Prompt Injection](https://mlsanity.com/paper/2306.08833/) (2023)

## Built on by (most cited first, 3 of 3)

- [Reasoning Hijacking: The Fragility of Reasoning Alignment in Large Language Models](https://mlsanity.com/paper/2601.10294/) (2026)
- [ChatGPT: Excellent Paper! Accept It. Editor: Imposter Found! Review Rejected](https://mlsanity.com/paper/2512.20405/) (2025)
- [Beyond the Editing Canvas: Evidence Divergence in OOXML-to-LLM Ingestion](https://mlsanity.com/paper/2608.25880/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are widely deployed in applications that accept user-submitted content, such as uploaded documents or pasted text, for tasks like summarization and question answering. In this paper, we identify a new class of attacks, prompt in content injection, where adversarial instructions are embedded in seemingly benign inputs. When processed by the LLM, these hidden prompts can manipulate outputs without user awareness or system compromise, leading to biased summaries, fabricated claims, or misleading suggestions. We demonstrate the feasibility of such attacks across popular platforms, analyze their root causes including prompt concatenation and insufficient input isolation, and discuss mitigation strategies. Our findings reveal a subtle yet practical threat in real-world LLM workflows.