# CrackedPDFs: A Controlled Benchmark for Hidden Prompt Injection in PDFs

arXiv: https://arxiv.org/abs/2607.19396 · HTML: https://mlsanity.com/paper/2607.19396/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 3 Jul 2026.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 14 of 14)

- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [Attention Tracker: Detecting Prompt Injection Attacks in LLMs](https://mlsanity.com/paper/2411.00348/) (2024)
- [Manipulating Multimodal Agents via Cross-Modal Prompt Injection](https://mlsanity.com/paper/2504.14348/) (2025)
- [On Training Robust PDF Malware Classifiers](https://mlsanity.com/paper/1904.03542/) (2019)
- [Prompt Injection Attacks on LLM Generated Reviews of Scientific Publications](https://mlsanity.com/paper/2509.10248/) (2025)
- [Invisible Prompts, Visible Threats: Malicious Font Injection in External Resources for Large Language Models](https://mlsanity.com/paper/2505.16957/) (2025)
- [When Skills Lie: Hidden-Comment Injection in LLM Agents](https://mlsanity.com/paper/2602.10498/) (2026)
- [Detection Method for Prompt Injection by Integrating Pre-trained Model and Heuristic Feature Engineering](https://mlsanity.com/paper/2506.06384/) (2025)
- [CAPTURE: Context-Aware Prompt Injection Testing and Robustness Enhancement](https://mlsanity.com/paper/2505.12368/) (2025)
- [Sanity Checks for Explanation Uncertainty](https://mlsanity.com/paper/2403.17212/) (2024)
- [PhantomLint: Principled Detection of Hidden LLM Prompts in Structured Documents](https://mlsanity.com/paper/2508.17884/) (2025)
- [Multilingual Hidden Prompt Injection Attacks on LLM-Based Academic Reviewing](https://mlsanity.com/paper/2512.23684/) (2025)
- [Zero-Shot Embedding Drift Detection: A Lightweight Defense Against Prompt Injections in LLMs](https://mlsanity.com/paper/2601.12359/) (2026)
- [Detecting Prompt Injection Attacks Against Application Using Classifiers](https://mlsanity.com/paper/2512.12583/) (2025)

## Abstract (arXiv)

Document-based LLM systems often flatten a PDF before guardrails inspect it. That step can discard evidence that an instruction was never visible to the user. We introduce CrackedPDFs, a controlled benchmark for hidden prompt injection in PDFs. The benchmark contains 29,322 generated PDFs from 4,983 base documents. It includes 9,774 injected files and 19,548 benign or matched-confounder files. We evaluate PromptGuard and a rule baseline. We also evaluate structural-only learned models and a sanitized hybrid detector. The evaluation uses held-out provenance splits and paired benign-confounder controls. It also uses label-shuffle checks and shortcut audits. On a 2,919-document held-out test set, the hybrid detector reaches 0.960 F1. ROC-AUC is 0.998 and PR-AUC is 0.997. On a balanced subset containing 973 injected PDFs and 973 matched benign confounders, the hybrid detector achieves 95.9% classification accuracy. Using score ordering, it ranks the injected member above its matched confounder in 100% of 973 pairs. PromptGuard has low recall when given extracted text only. Structural-only learned models are weak under paired controls. A text-only TF-IDF model reaches perfect held-out scores but fails shortcut audits. These results show that document-aware hybrid detection is useful under controlled paired evaluation. They do not show broad real-world robustness or reliable cross-family generalization.