mlsanityData to 7 Oct 2026

Seeing is not always believing: Benchmarking Human and Model Perception of AI-Generated Images

arXiv 2304.13023 ↗First posted 25 Apr 2023cs.AI cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

Photos serve as a way for humans to record what they experience in their daily lives, and they are often regarded as trustworthy sources of information. However, there is a growing concern that the advancement of artificial intelligence (AI) technology may produce fake photos, which can create confusion and diminish trust in photographs. This study aims to comprehensively evaluate agents for distinguishing state-of-the-art AI-generated visual content. Our study benchmarks both human capability and cutting-edge fake image detection AI algorithms, using a newly collected large-scale fake image dataset Fake2M. In our human perception evaluation, titled HPBench, we discovered that humans struggle significantly to distinguish real photos from AI-generated ones, with a misclassification rate of 38.7%. Along with this, we conduct the model capability of AI-Generated images detection evaluation MPBench and the top-performing model from MPBench achieves a 13% failure rate under the same setting used in the human evaluation. We hope that our study can raise awareness of the potential risks of AI-generated images and facilitate further research to prevent the spread of false information. More information can refer to https://github.com/Inf-imagine/Sentry.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 64 of its 84 references that are in the map. First posted on arXiv 25 Apr 2023.

Builds on

64 of 84 references.

References · 64

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2023GPT-4 Technical Reportcited by 18,132
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2020Denoising Diffusion Implicit Modelscited by 9,655
  11. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  12. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
38 more
  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022Generative Adversarial Networkscited by 5,603
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  10. 2022A ConvNet for the 2020scited by 3,545
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  12. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  13. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  14. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  15. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  16. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  17. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  18. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  19. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  20. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  21. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  22. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  23. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  24. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  25. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  26. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  27. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  28. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  29. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  30. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  31. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  32. 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 518
  33. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 501
  34. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  35. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  36. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 483
  37. 2021Deep Long-Tailed Learning: A Surveycited by 416
  38. 2019Detecting and Simulating Artifacts in GAN Fake Imagescited by 293

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 23; 2025: 23; 2026: 16.

65 papers.

Sort:

By area:

  1. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  2. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  3. 2023AGIQA-3K: An Open Database for AI-Generated Image Quality Assessmentcited by 142
  4. 2024Leveraging Representations from Intermediate Encoder-blocks for Synthetic Image Detectioncited by 88
  5. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 73
  6. 2025Is Artificial Intelligence Generated Image Detection a Solved Problem?cited by 49
  7. 2024Fake or JPEG? Revealing Common Biases in Generated Image Detection Datasetscited by 43
  8. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Modelscited by 36
  9. 2024MM-SpuBench: Towards Better Understanding of Spurious Biases in Multimodal LLMscited by 33
  10. 2024GAIA: Rethinking Action Quality Assessment for AI-Generated Videoscited by 30
  11. 2024AI-Face: A Million-Scale Demographically Annotated AI-Generated Face Dataset and Fairness Benchmarkcited by 28
  12. 2025Bridging the Gap Between Ideal and Real-world Evaluation: Benchmarking AI-Generated Image Detection in Challenging Scenarioscited by 24
38 more
  1. 2024Organic or Diffused: Can We Distinguish Human Art from AI-generated Images?cited by 21
  2. 2025Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Modelscited by 13
  3. 2025Survey on AI-Generated Media Detection: From Non-MLLM to MLLMcited by 12
  4. 2024ForgerySleuth: Empowering Multimodal Large Language Models for Image Manipulation Detectioncited by 11
  5. 2025AI-GenBench: A New Ongoing Benchmark for AI-Generated Image Detectioncited by 10
  6. 2024TextureCrop: Enhancing Synthetic Image Detection through Texture-based Croppingcited by 9
  7. 2025OpenFake: An Open Dataset and Platform Toward Real-World Deepfake Detectioncited by 9
  8. 2024Examining the Prevalence and Dynamics of AI-Generated Media in Art Subredditscited by 8
  9. 2025Navigating the Challenges of AI-Generated Image Detection in the Wild: What Truly Matters?cited by 8
  10. 2024Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunitiescited by 7
  11. 2024Unmasking Illusions: Understanding Human Perception of Audiovisual Deepfakescited by 7
  12. 2024Global-Local Image Perceptual Score (GLIPS): Evaluating Photorealistic Quality of AI-Generated Imagescited by 7
  13. 2025How good are humans at detecting AI-generated images? Learnings from an experimentcited by 7
  14. 2024AIGCs Confuse AI Too: Investigating and Explaining Synthetic Image-induced Hallucinations in Large Vision-Language Modelscited by 6
  15. 2024Conditioned Prompt-Optimization for Continual Deepfake Detectioncited by 6
  16. 2025Few-Shot Synthetic Image Attribution: Identifying Unseen Generators with Limited Samplescited by 6
  17. 2025PRPO: Paragraph-level Policy Optimization for Vision-Language Deepfake Detectioncited by 5
  18. 2026PGC: Peak-Guided Calibration for Generalizable AI-Generated Image Detectioncited by 5
  19. 2024The Adversarial AI-Art: Understanding, Generation, Detection, and Benchmarkingcited by 4
  20. 2024The Cat and Mouse Game: The Ongoing Arms Race Between Diffusion Models and Detection Methodscited by 4
  21. 2025A Crack in the Bark: Leveraging Public Knowledge to Remove Tree-Ring Watermarkscited by 4
  22. 2025Beyond Artificial Misalignment: Detecting and Grounding Semantic-Coordinated Multimodal Manipulationscited by 4
  23. 2025REVEAL: Reasoning-Enhanced Forensic Evidence Analysis for Explainable AI-Generated Image Detectioncited by 4
  24. 2026A Comprehensive Dataset for Human vs. AI Generated Image Detectioncited by 4
  25. 2024Detecting the Undetectable: Combining Kolmogorov-Arnold Networks and MLP for AI-Generated Image Detectioncited by 3
  26. 2024HyperDet: Generalizable Detection of Synthesized Images by Generating and Merging A Mixture of Hyper LoRAscited by 3
  27. 2025Stabilizing Self-Consuming Diffusion Models with Latent Space Filteringcited by 3
  28. 2024Generating Realistic X-ray Scattering Images Using Stable Diffusion and Human-in-the-loop Annotationscited by 2
  29. 2024Human vs. AI: A Novel Benchmark and a Comparative Study on the Detection of Generated Images and the Impact of Promptscited by 2
  30. 2025Generalized Design Choices for Deepfake Detectorscited by 2
  31. 2026Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detectioncited by 2
  32. 2026SSAFE: Simple and Strong AI-Generated Image Detection via Frozen Vision Encoderscited by 2
  33. 2025RAISE: Realness Assessment for Image Synthesis and Evaluationcited by 1
  34. 2025MiraGe: Multimodal Discriminative Representation Learning for Generalizable AI-Generated Image Detectioncited by 1
  35. 2025Mirage: Unveiling Hidden Artifacts in Synthetic Images with Large Vision-Language Modelscited by 1
  36. 2026Synthetic Image Detection with CLIP: Understanding and Assessing Predictive Cuescited by 1
  37. 2026From Masks to Pixels and Meaning: A New Taxonomy, Benchmark, and Metrics for VLM Image Tamperingcited by 1
  38. 2024VGMShield: Mitigating Misuse of Video Generative Modelscited by 0

50 most cited of 65 All 65 (JSON)