# Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images

arXiv: https://arxiv.org/abs/2304.00500 · HTML: https://mlsanity.com/paper/2304.00500/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 55 of its 83 references that are in the map.
- First posted on arXiv 2 Apr 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 55)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models](https://mlsanity.com/paper/2211.05105/) (2022)
- [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020)
- [Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors](https://mlsanity.com/paper/2203.13131/) (2022)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)

## Built on by (most cited first, 29 of 29)

- [Raising the Bar of AI-generated Image Detection with CLIP](https://mlsanity.com/paper/2312.00195/) (2023)
- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [Zero-Shot Detection of AI-Generated Images](https://mlsanity.com/paper/2409.15875/) (2024)
- [Generalizable Synthetic Image Detection via Language-guided Contrastive Learning](https://mlsanity.com/paper/2305.13800/) (2023)
- [Detecting Images Generated by Deep Diffusion Models using their Local Intrinsic Dimensionality](https://mlsanity.com/paper/2307.02347/) (2023)
- [DiffusionFace: Towards a Comprehensive Dataset for Diffusion-Based Face Forgery Analysis](https://mlsanity.com/paper/2403.18471/) (2024)
- [Faster Than Lies: Real-time Deepfake Detection using Binary Neural Networks](https://mlsanity.com/paper/2406.04932/) (2024)
- [Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities](https://mlsanity.com/paper/2407.20337/) (2024)
- [Bi-LORA: A Vision-Language Approach for Synthetic Image Detection](https://mlsanity.com/paper/2404.01959/) (2024)
- [Are CLIP features all you need for Universal Synthetic Image Origin Attribution?](https://mlsanity.com/paper/2408.09153/) (2024)
- [Synthetic Image Verification in the Era of Generative AI: What Works and What Isn't There Yet](https://mlsanity.com/paper/2405.00196/) (2024)
- [Exploring the Adversarial Robustness of CLIP for AI-generated Image Detection](https://mlsanity.com/paper/2407.19553/) (2024)
- [Forensic Self-Descriptions Are All You Need for Zero-Shot Detection, Open-Set Source Attribution, and Clustering of AI-generated Images](https://mlsanity.com/paper/2503.21003/) (2025)
- [Navigating the Challenges of AI-Generated Image Detection in the Wild: What Truly Matters?](https://mlsanity.com/paper/2507.10236/) (2025)
- [TextureCrop: Enhancing Synthetic Image Detection through Texture-based Cropping](https://mlsanity.com/paper/2407.15500/) (2024)
- [Detecting Origin Attribution for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2403.19653/) (2024)
- [Artifact Feature Purification for Cross-domain Detection of AI-generated Images](https://mlsanity.com/paper/2403.11172/) (2024)
- [Undercover Deepfakes: Detecting Fake Segments in Videos](https://mlsanity.com/paper/2305.06564/) (2023)
- [The Cat and Mouse Game: The Ongoing Arms Race Between Diffusion Models and Detection Methods](https://mlsanity.com/paper/2410.18866/) (2024)
- [Unveiling the Truth: Exploring Human Gaze Patterns in Fake Images](https://mlsanity.com/paper/2403.08933/) (2024)
- [Harlequin: Color-driven Generation of Synthetic Data for Referring Expression Comprehension](https://mlsanity.com/paper/2411.14807/) (2024)
- [Tex-ViT: A Generalizable, Robust, Texture-based dual-branch cross-attention deepfake detector](https://mlsanity.com/paper/2408.16892/) (2024)
- [FIDAVL: Fake Image Detection and Attribution using Vision-Language Model](https://mlsanity.com/paper/2409.03109/) (2024)
- [Provenance Detection for AI-Generated Images: Combining Perceptual Hashing, Homomorphic Encryption, and AI Detection Models](https://mlsanity.com/paper/2503.11195/) (2025)
- [Exploring Strengths and Weaknesses of Super-Resolution Attack in Deepfake Detection](https://mlsanity.com/paper/2410.04205/) (2024)
- [Modeling Human Responses to Multimodal AI Content](https://mlsanity.com/paper/2508.10769/) (2025)
- [Revisiting Deepfake Detection: Chronological Continual Learning and the Limits of Generalization](https://mlsanity.com/paper/2509.07993/) (2025)
- [Robustness of AI-Art Detectors under Generator Shift](https://mlsanity.com/paper/2608.11643/) (2026)

## Abstract (arXiv)

Recent advancements in diffusion models have enabled the generation of realistic deepfakes from textual prompts in natural language. While these models have numerous benefits across various sectors, they have also raised concerns about the potential misuse of fake images and cast new pressures on fake image detection. In this work, we pioneer a systematic study on deepfake detection generated by state-of-the-art diffusion models. Firstly, we conduct a comprehensive analysis of the performance of contrastive and classification-based visual features, respectively extracted from CLIP-based models and ResNet or ViT-based architectures trained on image classification datasets. Our results demonstrate that fake images share common low-level cues, which render them easily recognizable. Further, we devise a multimodal setting wherein fake images are synthesized by different textual captions, which are used as seeds for a generator. Under this setting, we quantify the performance of fake detection strategies and introduce a contrastive-based disentangling method that lets us analyze the role of the semantics of textual descriptions and low-level perceptual cues. Finally, we release a new dataset, called COCOFake, containing about 1.2M images generated from the original COCO image-caption pairs using two recent text-to-image diffusion models, namely Stable Diffusion v1.4 and v2.0.