# DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis

arXiv: https://arxiv.org/abs/2603.01433 · HTML: https://mlsanity.com/paper/2603.01433/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 47 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Builds on (most cited first, 21 of 21)

- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Towards Universal Fake Image Detectors that Generalize Across Generative Models](https://mlsanity.com/paper/2302.10174/) (2023)
- [DIRE for Diffusion-Generated Image Detection](https://mlsanity.com/paper/2303.09295/) (2023)
- [Media Forensics and DeepFakes: an overview](https://mlsanity.com/paper/2001.06564/) (2020)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization](https://mlsanity.com/paper/2212.10957/) (2022)
- [Image Manipulation Detection by Multi-View Multi-Scale Supervision](https://mlsanity.com/paper/2104.06832/) (2021)
- [ObjectFormer for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2203.14681/) (2022)
- [MIDV-2020: A Comprehensive Benchmark Dataset for Identity Document Analysis](https://mlsanity.com/paper/2107.00396/) (2021)
- [ForensicHub: A Unified Benchmark & Codebase for All-Domain Fake Image Detection and Localization](https://mlsanity.com/paper/2505.11003/) (2025)
- [Revisiting Tampered Scene Text Detection in the Era of Generative AI](https://mlsanity.com/paper/2407.21422/) (2024)
- [Toward Real Text Manipulation Detection: New Dataset and New Solution](https://mlsanity.com/paper/2312.06934/) (2023)
- [How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study](https://mlsanity.com/paper/2602.07814/) (2026)
- [Can Multi-modal (reasoning) LLMs work as deepfake detectors?](https://mlsanity.com/paper/2503.20084/) (2025)
- [Do Deepfake Detectors Work in Reality?](https://mlsanity.com/paper/2502.10920/) (2025)
- [Can Multi-modal (reasoning) LLMs detect document manipulation?](https://mlsanity.com/paper/2508.11021/) (2025)
- [FantasyID: A dataset for detecting digital manipulations of ID-documents](https://mlsanity.com/paper/2507.20808/) (2025)
- [TextShield-R1: Reinforced Reasoning for Tampered Text Detection](https://mlsanity.com/paper/2602.19828/) (2026)
- [Out of the box age estimation through facial imagery: A Comprehensive Benchmark of Vision-Language Models vs. out-of-the-box Traditional Architectures](https://mlsanity.com/paper/2602.07815/) (2026)
- [Toward Real-world Text Image Forgery Localization: Structured and Interpretable Data Synthesis](https://mlsanity.com/paper/2511.12658/) (2025)
- [LogicLens: Visual-Logical Co-Reasoning for Text-Centric Forgery Analysis](https://mlsanity.com/paper/2512.21482/) (2025)

## Built on by (most cited first, 4 of 4)

- [SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation](https://mlsanity.com/paper/2606.03348/) (2026)
- [FRAGMENT: Factorized Graph Representations for Document Generation and Editing via Entity-Aware Transformations](https://mlsanity.com/paper/2608.18679/) (2026)
- [Agents That Edit Documents: Measuring Agentic PDF Forgery Against a Non-Agentic Control](https://mlsanity.com/paper/2609.23953/) (2026)
- [Agentic Tool-Augmented Reasoning for Explainable Image Forgery Detection](https://mlsanity.com/paper/2609.39066/) (2026)

## Abstract (arXiv)

We present DOCFORGE-BENCH, the first unified zero-shot benchmark for document forgery detection, evaluating 14 methods across eight datasets spanning text tampering, receipt forgery, and identity document manipulation. Unlike fine-tuning-oriented evaluations such as ForensicHub [Du et al., 2025], DOCFORGE-BENCH applies all methods with their published pretrained weights and no domain adaptation -- a deliberate design choice that reflects the realistic deployment scenario where practitioners lack labeled document training data. Our central finding is a pervasive calibration failure invisible under single-threshold protocols: methods achieve moderate Pixel-AUC (>=0.76) yet near-zero Pixel-F1. This AUC-F1 gap is not a discrimination failure but a score-distribution shift: tampered regions occupy only 0.27-4.17% of pixels in document images -- an order of magnitude less than in natural image benchmarks -- making the standard tau=0.5 threshold catastrophically miscalibrated. Oracle-F1 is 2-10x higher than fixed-threshold Pixel-F1, confirming that calibration, not representation, is the bottleneck. A controlled calibration experiment validates this: adapting a single threshold on N=10 domain images recovers 39-55% of the Oracle-F1 gap, demonstrating that threshold adaptation -- not retraining -- is the key missing step for practical deployment. Overall, no evaluated method works reliably out-of-the-box on diverse document types, underscoring that document forgery detection remains an unsolved problem. We further note that all eight datasets predate the era of generative AI editing; benchmarks covering diffusion- and LLM-based document forgeries represent a critical open gap on the modern attack surface.