# GPT4o-Receipt: A Dataset and Human Study for AI-Generated Document Forensics

arXiv: https://arxiv.org/abs/2603.11442 · HTML: https://mlsanity.com/paper/2603.11442/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models](https://mlsanity.com/paper/2310.14566/) (2023)
- [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020)
- [DIRE for Diffusion-Generated Image Detection](https://mlsanity.com/paper/2303.09295/) (2023)
- [GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image](https://mlsanity.com/paper/2306.08571/) (2023)
- [On the detection of synthetic images generated by diffusion models](https://mlsanity.com/paper/2211.00680/) (2022)
- [Fourier Spectrum Discrepancies in Deep Network Generated Images](https://mlsanity.com/paper/1911.06465/) (2019)
- [TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization](https://mlsanity.com/paper/2212.10957/) (2022)
- [FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models](https://mlsanity.com/paper/2410.02761/) (2024)
- [Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensics](https://mlsanity.com/paper/2403.14077/) (2024)
- [Segment anything, from space?](https://mlsanity.com/paper/2304.13000/) (2023)
- [As Good As A Coin Toss: Human detection of AI-generated images, videos, audio, and audiovisual stimuli](https://mlsanity.com/paper/2403.16760/) (2024)
- [Revisiting Tampered Scene Text Detection in the Era of Generative AI](https://mlsanity.com/paper/2407.21422/) (2024)
- [How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study](https://mlsanity.com/paper/2602.07814/) (2026)
- [VisNumBench: Evaluating Number Sense of Multimodal Large Language Models](https://mlsanity.com/paper/2503.14939/) (2025)
- [Can Multi-modal (reasoning) LLMs work as deepfake detectors?](https://mlsanity.com/paper/2503.20084/) (2025)
- [Evaluating Numerical Reasoning in Text-to-Image Models](https://mlsanity.com/paper/2406.14774/) (2024)
- [Do Deepfake Detectors Work in Reality?](https://mlsanity.com/paper/2502.10920/) (2025)
- [Can Multi-modal (reasoning) LLMs detect document manipulation?](https://mlsanity.com/paper/2508.11021/) (2025)
- [Out of the box age estimation through facial imagery: A Comprehensive Benchmark of Vision-Language Models vs. out-of-the-box Traditional Architectures](https://mlsanity.com/paper/2602.07815/) (2026)
- [AIForge-Doc: A Benchmark for Detecting AI-Forged Tampering in Financial and Form Documents](https://mlsanity.com/paper/2602.20569/) (2026)
- [Can GPT tell us why these images are synthesized? Empowering Multimodal Large Language Models for Forensics](https://mlsanity.com/paper/2504.11686/) (2025)
- [Generation of Programmatic Rules for Document Forgery Detection Using Large Language Models](https://mlsanity.com/paper/2512.19228/) (2025)

## Built on by (most cited first, 4 of 4)

- [GPT-Image-2 in the Wild: A Twitter Dataset of Self-Reported AI-Generated Images from the First Week of Deployment](https://mlsanity.com/paper/2604.25370/) (2026)
- [TextRich: A Multi-Domain Benchmark for Detecting AI-Generated Text-Rich Images from GPT-Image-2](https://mlsanity.com/paper/2606.19259/) (2026)
- [ChatGPT Images 2.5 in the Wild: A Launch-Period Dataset and Detector Evaluation](https://mlsanity.com/paper/2609.15100/) (2026)
- [Agents That Edit Documents: Measuring Agentic PDF Forgery Against a Non-Agentic Control](https://mlsanity.com/paper/2609.23953/) (2026)

## Abstract (arXiv)

Can humans detect AI-generated financial documents better than machines? We present GPT4o-Receipt, a benchmark of 1,235 receipt images pairing GPT-4o-generated receipts with authentic ones from established datasets, evaluated by five state-of-the-art multimodal LLMs and a 30-annotator crowdsourced perceptual study. Our findings reveal a striking paradox: humans are better at seeing AI artifacts, yet worse at detecting AI documents. Human annotators exhibit the largest visual discrimination gap of any evaluator, yet their binary detection F1 falls well below Claude Sonnet 4 and below Gemini 2.5 Flash. This paradox resolves once the mechanism is understood: the dominant forensic signals in AI-generated receipts are arithmetic errors -- invisible to visual inspection but systematically verifiable by LLMs. Humans cannot perceive that a subtotal is incorrect; LLMs verify it in milliseconds. Beyond the human--LLM comparison, our five-model evaluation reveals dramatic performance disparities and calibration differences that render simple accuracy metrics insufficient for detector selection. GPT4o-Receipt, the evaluation framework, and all results are released publicly to support future research in AI document forensics.