mlsanityData to 7 Oct 2026

Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection

arXiv 2312.16649 ↗First posted 27 Dec 2023cs.CVcited by 180

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

In this paper, we study the problem of generalizable synthetic image detection, aiming to detect forgery images from diverse generative methods, e.g., GANs and diffusion models. Cutting-edge solutions start to explore the benefits of pre-trained models, and mainly follow the fixed paradigm of solely training an attached classifier, e.g., combining frozen CLIP-ViT with a learnable linear layer in UniFD. However, our analysis shows that such a fixed paradigm is prone to yield detectors with insufficient learning regarding forgery representations. We attribute the key challenge to the lack of forgery adaptation, and present a novel forgery-aware adaptive transformer approach, namely FatFormer. Based on the pre-trained vision-language spaces of CLIP, FatFormer introduces two core designs for the adaption to build generalized forgery representations. First, motivated by the fact that both image and frequency analysis are essential for synthetic image detection, we develop a forgery-aware adapter to adapt image features to discern and integrate local forgery traces within image and frequency domains. Second, we find that considering the contrastive objectives between adapted image features and text prompt embeddings, a previously overlooked aspect, results in a nontrivial generalization improvement. Accordingly, we introduce language-guided alignment to supervise the forgery adaptation with image and text prompts in FatFormer. Experiments show that, by coupling these two designs, our approach tuned on 4-class ProGAN data attains a remarkable detection performance, achieving an average of 98% accuracy to unseen GANs, and surprisingly generalizes to unseen diffusion models with 95% accuracy.

Key facts. Cited by 180 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 52 of its 71 references that are in the map. First posted on arXiv 27 Dec 2023.

Builds on

52 of 71 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023DIRE for Diffusion-Generated Image Detection16 Mar
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
38 more
  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  3. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  6. 2022Visual Prompt Tuningcited by 1,674
  7. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  8. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  9. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  10. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  11. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  12. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  13. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  14. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  15. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  16. 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 573
  17. 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 518
  18. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 501
  19. 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 483
  20. 2022Detecting Deepfakes with Self-Blended Imagescited by 332
  21. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  22. 2020Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributionscited by 278
  23. 2020What makes fake images detectable? Understanding properties that generalizecited by 272
  24. 2018Do GANs leave artificial fingerprints?cited by 188
  25. 2020Learning Self-Consistency for Deepfake Detectioncited by 179
  26. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  27. 2022CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-trainingcited by 159
  28. 2021Local Relation Learning for Face Forgery Detectioncited by 100
  29. 2023Detecting and Grounding Multi-Modal Media Manipulationcited by 98
  30. 2022FrePGAN: Robust Deepfake Detection Using Frequency-level Perturbationscited by 75
  31. 2020Wavelet Integrated CNNs for Noise-Robust Image Classificationcited by 60
  32. 2021BiHPF: Bilateral High-Pass Filters for Robust Deepfake Detectioncited by 60
  33. 2023Learning Mask-aware CLIP Representations for Zero-Shot Segmentationcited by 57
  34. 2021How Well Does Self-Supervised Pre-Training Perform with Streaming Data?cited by 29
  35. 2023CTP: Towards Vision-Language Continual Pretraining via Compatible Momentum Contrast and Topology Preservationcited by 29
  36. 2023FM-ViT: Flexible Modal Vision Transformers for Face Anti-Spoofingcited by 22
  37. 2023MA-ViT: Modality-Agnostic Vision Transformers for Face Anti-Spoofingcited by 18
  38. 2023Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulationcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 25; 2025: 72; 2026: 83.

180 papers.

Sort:

By area:

  1. 2024C2P-CLIP: Injecting Category Common Prompt in CLIP to Enhance Generalization in Deepfake Detectioncited by 111
  2. 2024Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detectioncited by 102
  3. 2024Improving Synthetic Image Detection Towards Generalization: An Image Transformation Perspectivecited by 88
  4. 2024A Bias-Free Training Paradigm for More General AI-generated Image Detectioncited by 67
  5. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 66
  6. 2025Dual Data Alignment Makes AI-Generated Image Detector Easier Generalizablecited by 58
  7. 2024Zero-Shot Detection of AI-Generated Imagescited by 50
  8. 2025CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AIcited by 42
  9. 2024On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detectioncited by 40
  10. 2024CFPL-FAS: Class Free Prompt Learning for Generalizable Face Anti-spoofingcited by 39
  11. 2025AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Modelscited by 33
  12. 2025Bridging the Gap Between Ideal and Real-world Evaluation: Benchmarking AI-Generated Image Detection in Challenging Scenarioscited by 24
38 more
  1. 2025All Patches Matter, More Patches Better: Enhance AI-Generated Image Detection via Panoptic Patch Learningcited by 23
  2. 2025ForensicHub: A Unified Benchmark & Codebase for All-Domain Fake Image Detection and Localizationcited by 22
  3. 2025Toward Generalizable Forgery Detection and Reasoningcited by 20
  4. 2025Deepfake Detection that Generalizes Across Benchmarkscited by 20
  5. 2024DeCLIP: Decoding CLIP representations for deepfake localizationcited by 17
  6. 2025Zooming In on Fakes: A Novel Dataset for Localized AI-Generated Image Detection with Forgery Amplification Approachcited by 16
  7. 2024Exploring Self-Supervised Vision Transformers for Deepfake Detection: A Comparative Analysiscited by 15
  8. 2024ForgeLens: Data-Efficient Forgery Focus for Generalizable Forgery Image Detectioncited by 14
  9. 2024Synthetic Image Verification in the Era of Generative AI: What Works and What Isn't There Yetcited by 13
  10. 2024A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authenticationcited by 12
  11. 2024Vulnerabilities in AI-generated Image Detection: The Challenge of Adversarial Attackscited by 12
  12. 2025Methods and Trends in Detecting AI-Generated Images: A Comprehensive Reviewcited by 12
  13. 2024MFCLIP: Multi-modal Fine-grained CLIP for Generalizable Diffusion Face Forgery Detectioncited by 11
  14. 2025Dual Frequency Branch Framework with Reconstructed Sliding Windows Attention for AI-Generated Image Detectioncited by 10
  15. 2025Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detectioncited by 10
  16. 2025Brought a Gun to a Knife Fight: Modern VFM Baselines Outgun Specialized Detectors on In-the-Wild AI Image Detectioncited by 10
  17. 2026DNA: Uncovering Universal Latent Forgery Knowledgecited by 10
  18. 2025FerretNet: Efficient Synthetic Image Detection via Local Pixel Dependenciescited by 9
  19. 2025Training-free Detection of AI-generated images via Cropping Robustnesscited by 9
  20. 2025Towards Generalizable AI-Generated Image Detection via Image-Adaptive Prompt Learningcited by 8
  21. 2025RAIDX: A Retrieval-Augmented Generation and GRPO Reinforcement Learning Framework for Explainable Deepfake Detectioncited by 8
  22. 2025Scaling Up AI-Generated Image Detection with Generator-Aware Prototypescited by 8
  23. 2026Preserving Forgery Artifacts: AI-Generated Video Detection at Native Scalecited by 8
  24. 2025NS-Net: Decoupling CLIP Semantic Information through NULL-Space for Generalizable AI-Generated Image Detectioncited by 7
  25. 2026Venus-DeFakerOne: Unified Fake Image Detection & Localizationcited by 7
  26. 2025E2LVLM:Evidence-Enhanced Large Vision-Language Model for Multimodal Out-of-Context Misinformation Detectioncited by 6
  27. 2025Recover and Match: Open-Vocabulary Multi-Label Recognition through Knowledge-Constrained Optimal Transportcited by 6
  28. 2025FantasyID: A dataset for detecting digital manipulations of ID-documentscited by 6
  29. 2026Simplicity Prevails: The Emergence of Generalizable AIGI Detection in Visual Foundation Modelscited by 6
  30. 2026EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detectioncited by 6
  31. 2025DATA: Multi-Disentanglement based Contrastive Learning for Open-World Semi-Supervised Deepfake Attributioncited by 5
  32. 2025Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detectioncited by 5
  33. 2025Pay Less Attention to Deceptive Artifacts: Robust Detection of Compressed Deepfakes on Online Social Networkscited by 5
  34. 2025Detecting Generated Images by Fitting Natural Image Distributionscited by 5
  35. 2025AI-Generated Image Detection: An Empirical Study and Future Research Directionscited by 5
  36. 2025DINO-Detect: A Simple yet Effective Framework for Blur-Robust AI-Generated Image Detectioncited by 5
  37. 2026PGC: Peak-Guided Calibration for Generalizable AI-Generated Image Detectioncited by 5
  38. 2024Dealing with Synthetic Data Contamination in Online Continual Learningcited by 4

50 most cited of 180 All 180 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Generalizable Synthetic Image Detection via Language-guided Contrastive Learning
  2. 2025Unlocking the Forgery Detection Potential of Vanilla MLLMs: A Novel Training-Free Pipeline
  3. 2026Universal Anti-forensics Attack against Image Forgery Detection via Multi-modal Guidance
  4. 2024Are CLIP features all you need for Universal Synthetic Image Origin Attribution?
  5. 2025AdaptPrompt: Parameter-Efficient Adaptation of VLMs for Generalizable Deepfake Detection
  6. 2026ForensicFormer: Hierarchical Multi-Scale Reasoning for Cross-Domain Image Forgery Detection
  7. 2023Intriguing properties of synthetic images: from generative adversarial networks to diffusion models
  8. 2025Identity-Aware Vision-Language Model for Explainable Face Forgery Detection