# Can Multi-modal (reasoning) LLMs detect document manipulation?

arXiv: https://arxiv.org/abs/2508.11021 · HTML: https://mlsanity.com/paper/2508.11021/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 6 of its 27 references that are in the map.
- First posted on arXiv 14 Aug 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Builds on (most cited first, 6 of 6)

- [ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis](https://mlsanity.com/paper/2103.05630/) (2021)
- [MIDV-2020: A Comprehensive Benchmark Dataset for Identity Document Analysis](https://mlsanity.com/paper/2107.00396/) (2021)
- [Can Multi-modal (reasoning) LLMs work as deepfake detectors?](https://mlsanity.com/paper/2503.20084/) (2025)
- [Automated Extraction of Energy Systems Information from Remotely Sensed Data: A Review and Analysis](https://mlsanity.com/paper/2202.12939/) (2022)
- [DF2023: The Digital Forensics 2023 Dataset for Image Forgery Detection](https://mlsanity.com/paper/2503.22417/) (2025)
- [Image Generation and Learning Strategy for Deep Document Forgery Detection](https://mlsanity.com/paper/2311.03650/) (2023)

## Built on by (most cited first, 7 of 7)

- [Out of the box age estimation through facial imagery: A Comprehensive Benchmark of Vision-Language Models vs. out-of-the-box Traditional Architectures](https://mlsanity.com/paper/2602.07815/) (2026)
- [AIForge-Doc: A Benchmark for Detecting AI-Forged Tampering in Financial and Form Documents](https://mlsanity.com/paper/2602.20569/) (2026)
- [DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis](https://mlsanity.com/paper/2603.01433/) (2026)
- [GPT4o-Receipt: A Dataset and Human Study for AI-Generated Document Forensics](https://mlsanity.com/paper/2603.11442/) (2026)
- [When the Forger Is the Judge: GPT-Image-2 Cannot Recognize Its Own Faked Documents](https://mlsanity.com/paper/2604.25213/) (2026)
- [A Synthetic Eye Movement Dataset for Script Reading Detection: Real Trajectory Replay on a 3D Simulator](https://mlsanity.com/paper/2604.05475/) (2026)
- [Agents That Edit Documents: Measuring Agentic PDF Forgery Against a Non-Agentic Control](https://mlsanity.com/paper/2609.23953/) (2026)

## Abstract (arXiv)

Document fraud poses a significant threat to industries reliant on secure and verifiable documentation, necessitating robust detection mechanisms. This study investigates the efficacy of state-of-the-art multi-modal large language models (LLMs)-including OpenAI O1, OpenAI 4o, Gemini Flash (thinking), Deepseek Janus, Grok, Llama 3.2 and 4, Qwen 2 and 2.5 VL, Mistral Pixtral, and Claude 3.5 and 3.7 Sonnet-in detecting fraudulent documents. We benchmark these models against each other and prior work on document fraud detection techniques using a standard dataset with real transactional documents. Through prompt optimization and detailed analysis of the models' reasoning processes, we evaluate their ability to identify subtle indicators of fraud, such as tampered text, misaligned formatting, and inconsistent transactional sums. Our results reveal that top-performing multi-modal LLMs demonstrate superior zero-shot generalization, outperforming conventional methods on out-of-distribution datasets, while several vision LLMs exhibit inconsistent or subpar performance. Notably, model size and advanced reasoning capabilities show limited correlation with detection accuracy, suggesting task-specific fine-tuning is critical. This study underscores the potential of multi-modal LLMs in enhancing document fraud detection systems and provides a foundation for future research into interpretable and scalable fraud mitigation strategies.