# Human vs. AI: A Novel Benchmark and a Comparative Study on the Detection of Generated Images and the Impact of Prompts

arXiv: https://arxiv.org/abs/2412.09715 · HTML: https://mlsanity.com/paper/2412.09715/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image](https://mlsanity.com/paper/2306.08571/) (2023)
- [On the detection of synthetic images generated by diffusion models](https://mlsanity.com/paper/2211.00680/) (2022)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models](https://mlsanity.com/paper/2210.06998/) (2022)
- [Are GAN generated images easy to detect? A critical analysis of the state-of-the-art](https://mlsanity.com/paper/2104.02617/) (2021)
- [CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images](https://mlsanity.com/paper/2303.14126/) (2023)
- [Evading Deepfake-Image Detectors with White- and Black-Box Attacks](https://mlsanity.com/paper/2004.00622/) (2020)
- [Seeing is not always believing: Benchmarking Human and Model Perception of AI-Generated Images](https://mlsanity.com/paper/2304.13023/) (2023)
- [Online Detection of AI-Generated Images](https://mlsanity.com/paper/2310.15150/) (2023)
- [As Good As A Coin Toss: Human detection of AI-generated images, videos, audio, and audiovisual stimuli](https://mlsanity.com/paper/2403.16760/) (2024)
- [Misleading Deep-Fake Detection with GAN Fingerprints](https://mlsanity.com/paper/2205.12543/) (2022)
- [Global-Local Image Perceptual Score (GLIPS): Evaluating Photorealistic Quality of AI-Generated Images](https://mlsanity.com/paper/2405.09426/) (2024)

## Built on by (most cited first, 2 of 2)

- [Prompt and Circumstances: Evaluating the Efficacy of Human Prompt Inference in AI-Generated Art](https://mlsanity.com/paper/2601.17379/) (2026)
- [Navigating the Open-Source Model Ecosystem: An Empirical Study of Creator Practices in Artistic Image Generation](https://mlsanity.com/paper/2607.10538/) (2026)

## Abstract (arXiv)

With the advent of publicly available AI-based text-to-image systems, the process of creating photorealistic but fully synthetic images has been largely democratized. This can pose a threat to the public through a simplified spread of disinformation. Machine detectors and human media expertise can help to differentiate between AI-generated (fake) and real images and counteract this danger. Although AI generation models are highly prompt-dependent, the impact of the prompt on the fake detection performance has rarely been investigated yet. This work therefore examines the influence of the prompt's level of detail on the detectability of fake images, both with an AI detector and in a user study. For this purpose, we create a novel dataset, COCOXGEN, which consists of real photos from the COCO dataset as well as images generated with SDXL and Fooocus using prompts of two standardized lengths. Our user study with 200 participants shows that images generated with longer, more detailed prompts are detected significantly more easily than those generated with short prompts. Similarly, an AI-based detection model achieves better performance on images generated with longer prompts. However, humans and AI models seem to pay attention to different details, as we show in a heat map analysis.