# Distinguishing Natural and Computer-Generated Images using Multi-Colorspace fused EfficientNet

arXiv: https://arxiv.org/abs/2110.09428 · HTML: https://mlsanity.com/paper/2110.09428/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 66 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 24 of 24)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos](https://mlsanity.com/paper/1810.11215/) (2018)
- [Detecting GAN generated Fake Images using Co-occurrence Matrices](https://mlsanity.com/paper/1903.06836/) (2019)
- [Exposing GAN-generated Faces Using Inconsistent Corneal Specular Highlights](https://mlsanity.com/paper/2009.11924/) (2020)
- [Incremental learning for the detection and classification of GAN-generated images](https://mlsanity.com/paper/1910.01568/) (2019)
- [Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection](https://mlsanity.com/paper/1703.04615/) (2017)
- [CNN Detection of GAN-Generated Face Images based on Cross-Band Co-occurrences Analysis](https://mlsanity.com/paper/2007.12909/) (2020)
- [Detecting CNN-Generated Facial Images in Real-World Scenarios](https://mlsanity.com/paper/2005.05632/) (2020)
- [ColorNet: Investigating the importance of color spaces for image classification](https://mlsanity.com/paper/1902.00267/) (2019)
- [Detection, Attribution and Localization of GAN Generated Images](https://mlsanity.com/paper/2007.10466/) (2020)
- [Identification of Deep Network Generated Images Using Disparities in Color Components](https://mlsanity.com/paper/1808.07276/) (2018)
- [Humans Are Easily Fooled by Digital Images](https://mlsanity.com/paper/1509.05301/) (2015)
- [Exposing Computer Generated Images by Using Deep Convolutional Neural Networks](https://mlsanity.com/paper/1711.10394/) (2017)

## Built on by (most cited first, 4 of 4)

- [Deep Image Fingerprint: Towards Low Budget Synthetic Image Detection and Model Lineage Analysis](https://mlsanity.com/paper/2303.10762/) (2023)
- [Expressive Talking Head Video Encoding in StyleGAN2 Latent-Space](https://mlsanity.com/paper/2203.14512/) (2022)
- [Towards Exploring Fairness in Visual Transformer based Natural and GAN Image Detection Systems](https://mlsanity.com/paper/2310.12076/) (2023)
- [Swin Transformer for Robust Differentiation of Real and Synthetic Images: Intra- and Inter-Dataset Analysis](https://mlsanity.com/paper/2409.04734/) (2024)

## Abstract (arXiv)

The problem of distinguishing natural images from photo-realistic computer-generated ones either addresses natural images versus computer graphics or natural images versus GAN images, at a time. But in a real-world image forensic scenario, it is highly essential to consider all categories of image generation, since in most cases image generation is unknown. We, for the first time, to our best knowledge, approach the problem of distinguishing natural images from photo-realistic computer-generated images as a three-class classification task classifying natural, computer graphics, and GAN images. For the task, we propose a Multi-Colorspace fused EfficientNet model by parallelly fusing three EfficientNet networks that follow transfer learning methodology where each network operates in different colorspaces, RGB, LCH, and HSV, chosen after analyzing the efficacy of various colorspace transformations in this image forensics problem. Our model outperforms the baselines in terms of accuracy, robustness towards post-processing, and generalizability towards other datasets. We conduct psychophysics experiments to understand how accurately humans can distinguish natural, computer graphics, and GAN images where we could observe that humans find difficulty in classifying these images, particularly the computer-generated images, indicating the necessity of computational algorithms for the task. We also analyze the behavior of our model through visual explanations to understand salient regions that contribute to the model's decision making and compare with manual explanations provided by human participants in the form of region markings, where we could observe similarities in both the explanations indicating the powerful nature of our model to take the decisions meaningfully.