mlsanityData to 7 Oct 2026

Real, fake and synthetic faces -- does the coin have three sides?

arXiv 2404.01878 ↗First posted 2 Apr 2024cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

With the ever-growing power of generative artificial intelligence, deepfake and artificially generated (synthetic) media have continued to spread online, which creates various ethical and moral concerns regarding their usage. To tackle this, we thus present a novel exploration of the trends and patterns observed in real, deepfake and synthetic facial images. The proposed analysis is done in two parts: firstly, we incorporate eight deep learning models and analyze their performances in distinguishing between the three classes of images. Next, we look to further delve into the similarities and differences between these three sets of images by investigating their image properties both in the context of the entire image as well as in the context of specific regions within the image. ANOVA test was also performed and provided further clarity amongst the patterns associated between the images of the three classes. From our findings, we observe that the investigated deeplearning models found it easier to detect synthetic facial images, with the ViT Patch-16 model performing best on this task with a class-averaged sensitivity, specificity, precision, and accuracy of 97.37%, 98.69%, 97.48%, and 98.25%, respectively. This observation was supported by further analysis of various image properties. We saw noticeable differences across the three category of images. This analysis can help us build better algorithms for facial image generation, and also shows that synthetic, deepfake and real face images are indeed three different classes.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 51 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

36 of 51 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2022Generative Adversarial Networkscited by 5,603
  9. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  10. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  11. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  12. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
24 more
  1. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  2. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  3. 2016Generative Adversarial Text to Image Synthesiscited by 735
  4. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  5. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  6. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  7. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  8. 2017AttGAN: Facial Attribute Editing by Only Changing What You Wantcited by 241
  9. 2020DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detectioncited by 231
  10. 2019Controllable Text-to-Image Generationcited by 187
  11. 2020DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesiscited by 161
  12. 2016Invertible Conditional GANs for image editingcited by 118
  13. 2017Fader Networks: Manipulating Images by Sliding Attributescited by 118
  14. 2016Learning What and Where to Drawcited by 105
  15. 2021Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forwardcited by 82
  16. 2020DeepFake Detection by Analyzing Convolutional Tracescited by 81
  17. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 73
  18. 2022MARLIN: Masked Autoencoder for facial video Representation LearnINgcited by 55
  19. 2018Image-to-Image Translation via Group-wise Deep Whitening-and-Coloring Transformationcited by 49
  20. 2016Learning Residual Images for Face Attribute Manipulationcited by 40
  21. 2016Deep Identity-aware Transfer of Facial Attributescited by 13
  22. 2018PCGAN: Partition-Controlled Human Image Generationcited by 6
  23. 2020Synthesis and Edition of Ultrasound Images via Sketch Guided Progressive Growing GANscited by 6
  24. 2023On the Vulnerability of DeepFake Detectors to Attacks Generated by Denoising Diffusion Modelscited by 6

Built on by

Not cited yet.