# Agents That Edit Documents: Measuring Agentic PDF Forgery Against a Non-Agentic Control

arXiv: https://arxiv.org/abs/2609.23953 · HTML: https://mlsanity.com/paper/2609.23953/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 20 Sep 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 24 of 24)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models](https://mlsanity.com/paper/2408.08926/) (2024)
- [Hierarchical Fine-Grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2303.17111/) (2023)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization](https://mlsanity.com/paper/2212.10957/) (2022)
- [Evaluating Frontier Models for Dangerous Capabilities](https://mlsanity.com/paper/2403.13793/) (2024)
- [MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection](https://mlsanity.com/paper/2112.08935/) (2021)
- [FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models](https://mlsanity.com/paper/2410.02761/) (2024)
- [SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model](https://mlsanity.com/paper/2412.04292/) (2024)
- [IMDL-BenCo: A Comprehensive Benchmark and Codebase for Image Manipulation Detection & Localization](https://mlsanity.com/paper/2406.10580/) (2024)
- [ForensicHub: A Unified Benchmark & Codebase for All-Domain Fake Image Detection and Localization](https://mlsanity.com/paper/2505.11003/) (2025)
- [ForgeryGPT: A Multimodal LLM for Interpretable Image Forgery Detection and Localization](https://mlsanity.com/paper/2410.10238/) (2024)
- [Omni-IML: Towards Unified Image Manipulation Localization](https://mlsanity.com/paper/2411.14823/) (2024)
- [Can Multi-modal (reasoning) LLMs detect document manipulation?](https://mlsanity.com/paper/2508.11021/) (2025)
- [AIForge-Doc: A Benchmark for Detecting AI-Forged Tampering in Financial and Form Documents](https://mlsanity.com/paper/2602.20569/) (2026)
- [Weakly-Supervised Image Forgery Localization via Vision-Language Collaborative Reasoning Framework](https://mlsanity.com/paper/2508.01338/) (2025)
- [DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis](https://mlsanity.com/paper/2603.01433/) (2026)
- [GPT4o-Receipt: A Dataset and Human Study for AI-Generated Document Forensics](https://mlsanity.com/paper/2603.11442/) (2026)
- [When the Forger Is the Judge: GPT-Image-2 Cannot Recognize Its Own Faked Documents](https://mlsanity.com/paper/2604.25213/) (2026)
- [ChatGPT Images 2.5 on Forgery Tasks: Testing Advertised Improvements Against Known Answers](https://mlsanity.com/paper/2609.13617/) (2026)

## Abstract (arXiv)

AI agents that carry a multi-step computer task through on their own became ordinary tools in the past year, and the same autonomy is available to anyone whose task is harmful. We ask what that means for a relying party -- an insurer, a lender, an auditor -- whose evidence is a filed PDF. AgentForge-Bench measures how reliably an off-the-shelf coding agent, driving one of seven open-weight models with a shell and the stock Python PDF stack, alters one dollar amount, date or address in a real filed financial document from a single sentence of intent, graded by rules rather than by a model. Across 1,750 cells, 1,419 (81.1%) satisfy the verifier, and 808 (46.2%) also survive every stricter filter: visible, localized, typeface-matched, original value gone document-wide. A deterministic script with no model in it solves 98 of the 125 documents; the agents solve 124, and none the script solves alone. Agents misreport 41% of their wrong edits as done, no model refused, and the cheapest verified forgery costs 2.4 cents. The raw rate overstates the threat by about a factor of two; the strict rate is still large.