# FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models

arXiv: https://arxiv.org/abs/2404.13306 · HTML: https://mlsanity.com/paper/2404.13306/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 94 of its 133 references that are in the map.
- First posted on arXiv 20 Apr 2024.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Builds on (most cited first, 30 of 94)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)

## Built on by (most cited first, 30 of 36)

- [MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs](https://mlsanity.com/paper/2411.15296/) (2024)
- [A Bias-Free Training Paradigm for More General AI-generated Image Detection](https://mlsanity.com/paper/2412.17671/) (2024)
- [LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models](https://mlsanity.com/paper/2410.09732/) (2024)
- [X2-DFD: A framework for eXplainable and eXtendable Deepfake Detection](https://mlsanity.com/paper/2410.06126/) (2024)
- [FakeScope: Large Multimodal Expert Model for Transparent AI-Generated Image Forensics](https://mlsanity.com/paper/2503.24267/) (2025)
- [Measuring the Measurers: Quality Evaluation of Hallucination Benchmarks for Large Vision-Language Models](https://mlsanity.com/paper/2406.17115/) (2024)
- [Semantic Visual Anomaly Detection and Reasoning in AI-Generated Images](https://mlsanity.com/paper/2510.10231/) (2025)
- [Fake-HR1: Rethinking Reasoning of Vision Language Model for Synthetic Image Detection](https://mlsanity.com/paper/2602.10042/) (2026)
- [Explainable Synthetic Image Detection through Diffusion Timestep Ensembling](https://mlsanity.com/paper/2503.06201/) (2025)
- [EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detection](https://mlsanity.com/paper/2603.17343/) (2026)
- [PRPO: Paragraph-level Policy Optimization for Vision-Language Deepfake Detection](https://mlsanity.com/paper/2509.26272/) (2025)
- [From Evidence to Verdict: An Agent-Based Forensic Framework for AI-Generated Image Detection](https://mlsanity.com/paper/2511.00181/) (2025)
- [REVEAL: Reasoning-Enhanced Forensic Evidence Analysis for Explainable AI-Generated Image Detection](https://mlsanity.com/paper/2511.23158/) (2025)
- [Locate-Then-Examine: Grounded Region Reasoning Improves Detection of AI-Generated Images](https://mlsanity.com/paper/2510.04225/) (2025)
- [FakeVLM-R1: Internalizing Physical Laws via CoT for Synthetic Image Detection](https://mlsanity.com/paper/2605.30062/) (2026)
- [Unveiling Perceptual Artifacts: A Fine-Grained Benchmark for Interpretable AI-Generated Image Detection](https://mlsanity.com/paper/2601.19430/) (2026)
- [AgentFoX: LLM Agent-Guided Fusion with eXplainability for AI-Generated Image Detection](https://mlsanity.com/paper/2603.23115/) (2026)
- [EditScout: Locating Forged Regions from Diffusion-based Edited Images with Multimodal LLM](https://mlsanity.com/paper/2412.03809/) (2024)
- [SAGA: Source Attribution of Generative AI Videos](https://mlsanity.com/paper/2511.12834/) (2025)
- [GRRE: Leveraging G-Channel Removed Reconstruction Error for Robust Detection of AI-Generated Images](https://mlsanity.com/paper/2601.02709/) (2026)
- [Exploiting the Final Component of Generator Architectures for AI-Generated Image Detection](https://mlsanity.com/paper/2601.20461/) (2026)
- [ArtifactLens: Hundreds of Labels Are Enough for Artifact Detection with VLMs](https://mlsanity.com/paper/2602.09475/) (2026)
- [GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection?](https://mlsanity.com/paper/2603.18625/) (2026)
- [XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection](https://mlsanity.com/paper/2607.03562/) (2026)
- [Unmasking Synthetic Realities in Generative AI: A Comprehensive Review of Adversarially Robust Deepfake Detection Systems](https://mlsanity.com/paper/2507.21157/) (2025)
- [Don't Guess, Escalate: Towards Explainable Uncertainty-Calibrated AI Forensic Agents](https://mlsanity.com/paper/2512.16614/) (2025)
- [Generalizable Detection of AI Generated Images with Large Models and Fuzzy Decision Tree](https://mlsanity.com/paper/2603.28508/) (2026)
- [Multi-Feature Fusion Approach for Generative AI Images Detection](https://mlsanity.com/paper/2603.29788/) (2026)
- [Beyond Semantics: Uncovering the Physics of Fakes via Universal Physical Descriptors for Cross-Modal Synthetic Detection](https://mlsanity.com/paper/2604.04608/) (2026)
- [SynMulti: Synthetic-to-Real Learning for Multimodal Video Understanding](https://mlsanity.com/paper/2604.12335/) (2026)

## Abstract (arXiv)

The ability to distinguish whether an image is generated by artificial intelligence (AI) is a crucial ingredient in human intelligence, usually accompanied by a complex and dialectical forensic and reasoning process. However, current fake image detection models and databases focus on binary classification without understandable explanations for the general populace. This weakens the credibility of authenticity judgment and may conceal potential model biases. Meanwhile, large multimodal models (LMMs) have exhibited immense visual-text capabilities on various tasks, bringing the potential for explainable fake image detection. Therefore, we pioneer the probe of LMMs for explainable fake image detection by presenting a multimodal database encompassing textual authenticity descriptions, the FakeBench. For construction, we first introduce a fine-grained taxonomy of generative visual forgery concerning human perception, based on which we collect forgery descriptions in human natural language with a human-in-the-loop strategy. FakeBench examines LMMs with four evaluation criteria: detection, reasoning, interpretation and fine-grained forgery analysis, to obtain deeper insights into image authenticity-relevant capabilities. Experiments on various LMMs confirm their merits and demerits in different aspects of fake image detection tasks. This research presents a paradigm shift towards transparency for the fake image detection area and reveals the need for greater emphasis on forensic elements in visual-language research and AI risk control. FakeBench will be available at https://github.com/Yixuan423/FakeBench.