mlsanityData to 14 Oct 2026

Frontier vision-language models have overtaken young adults at detecting AI-generated portraits -- but not their calibration

arXiv 2608.30210 ↗First posted 31 Aug 2026cs.HC cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

AI image generators now create face portraits that are hard to tell from real photographs. Vision-language models (VLMs) are increasingly proposed to flag such images. We benchmarked 19 VLMs on the same 198 face portraits -- real photographs and identity-matched ChatGPT-4o and Imagen 3 versions -- under the same task as our earlier study of 1,667 adults (85% correct overall; accuracy fell steeply with age). The June-2026 cohort of 14 models only matched adults in their 20s-30s. Four weeks later the ceiling broke. Among five July-2026 releases under the identical protocol, gpt-5.6-sol reached 92.8% balanced accuracy (five-draw mean 92.1%), clearly above adults in their 20s (88.5%), and claude-fable-5 detected every AI image while averaging 91.9%. Model sensitivity now exceeds young adults decisively (d' up to 3.4 versus ~ 2.4). What has not been overtaken is human calibration. Model criteria spread from c = -1.10 to +1.45 while humans sit near zero at every age; both new leaders are biased (+0.44, -0.97), and only a few mid-ranked models approach the human balance. Changing the labelled examples still flipped about one answer in four. The best machines now out-see young adults here, without matching the human balance between suspicion and trust.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 45 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

30 of 45 references.

References · 30

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  6. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  7. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  8. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  9. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  10. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
  11. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
18 more
  1. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 490
  2. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  3. 2022Using cognitive psychology to understand GPT-3cited by 398
  4. 2022On the detection of synthetic images generated by diffusion modelscited by 283
  5. 2021Partial success in closing the gap between human and machine visioncited by 185
  6. 2024Are self-explanations from Large Language Models faithful?cited by 101
  7. 2024Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensicscited by 69
  8. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 68
  9. 2021Deepfake Detection by Human Crowds, Machines, and Machine-informed Crowdscited by 58
  10. 2024What Makes Multimodal In-Context Learning Work?cited by 56
  11. 2024LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Modelscited by 51
  12. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Modelscited by 36
  13. 2021More Real than Real: A Study on Human Visual Perception of Synthetic Facescited by 23
  14. 2023A Representative Study on Human Detection of Artificially Generated Media Across Countriescited by 22
  15. 2022Testing Human Ability To Detect Deepfake Images of Human Facescited by 15
  16. 2021A Study of the Human Perception of Synthetic Facescited by 14
  17. 2024Characteristics and prevalence of fake social media profiles with AI-generated facescited by 14
  18. 2026Human Factors in Detecting AI-Generated Portraits: Age, Sex, Device, and Confidencecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Out of the box age estimation through facial imagery: A Comprehensive Benchmark of Vision-Language Models vs. out-of-the-box Traditional Architectures
  2. 2024A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models
  3. 2025DemoBias: An Empirical Study to Trace Demographic Biases in Vision Foundation Models
  4. 2026Newer Is Not Fairer: Gender Stereotyping in Text-to-Image AI Across Model Generations
  5. 2026Evolution of Accuracy and Visual-Cognitive Errors in a Decade of Vision-Language AI Models
  6. 2026Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness
  7. 2026VLAgeBench: Benchmarking Large Vision-Language Models for Zero-Shot Human Age Estimation
  8. 2024FaceScore: Benchmarking and Enhancing Face Quality in Human Generation