mlsanityData to 7 Oct 2026

AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors

arXiv 2310.17419 ↗First posted 26 Oct 2023cs.CVcited by 64

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Deep generative models can create remarkably photorealistic fake images while raising concerns about misinformation and copyright infringement, known as deepfake threats. Deepfake detection technique is developed to distinguish between real and fake images, where the existing methods typically learn classifiers in the image domain or various feature domains. However, the generalizability of deepfake detection against emerging and more advanced generative models remains challenging. In this paper, being inspired by the zero-shot advantages of Vision-Language Models (VLMs), we propose a novel approach called AntifakePrompt, using VLMs (e.g., InstructBLIP) and prompt tuning techniques to improve the deepfake detection accuracy over unseen data. We formulate deepfake detection as a visual question answering problem, and tune soft prompts for InstructBLIP to answer the real/fake information of a query image. We conduct full-spectrum experiments on datasets from a diversity of 3 held-in and 20 held-out generative models, covering modern text-to-image generation, image editing and adversarial image attacks. These testing datasets provide useful benchmarks in the realm of deepfake detection for further research. Moreover, results demonstrate that (1) the deepfake detection accuracy can be significantly and consistently improved (from 71.06% to 92.11%, in average accuracy over unseen domains) using pretrained vision-language models with prompt tuning; (2) our superior performance is at less cost of training data and trainable parameters, resulting in an effective and efficient solution for deepfake detection. Code and models can be found at https://github.com/nctu-eva-lab/AntifakePrompt.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 73 of its 99 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

73 of 99 references.

References · 73

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2020Denoising Diffusion Implicit Modelscited by 9,655
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  10. 2023Visual Instruction Tuningcited by 7,792
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
38 more
  1. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  13. 2016Improved Techniques for Training GANscited by 2,938
  14. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  15. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  16. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  17. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  18. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  19. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  20. 2019Towards VQA Models That Can Readcited by 1,696
  21. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  22. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  23. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  24. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  25. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  26. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  27. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 842
  28. 2018Self-Attention Generative Adversarial Networkscited by 809
  29. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  30. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  31. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  32. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  33. 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 573
  34. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 550
  35. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 501
  36. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  37. 2023DIRE for Diffusion-Generated Image Detectioncited by 386
  38. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 376

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 31; 2026: 19.

64 papers.

Sort:

By area:

  1. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 124
  2. 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Modelcited by 107
  3. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 73
  4. 2024A Bias-Free Training Paradigm for More General AI-generated Image Detectioncited by 67
  5. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 66
  6. 2024FIRE: Robust Detection of Diffusion-Generated Images via Frequency-Guided Reconstruction Errorcited by 46
  7. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Modelscited by 36
  8. 2025AI-Generated Video Detection via Perceptual Straighteningcited by 34
  9. 2025AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Modelscited by 33
  10. 2025BusterX: MLLM-Powered AI-Generated Video Forgery Detection and Explanationcited by 30
  11. 2025BusterX++: Towards Unified Cross-Modal AI-Generated Content Detection and Explanation with MLLMcited by 28
  12. 2024Safety Alignment for Vision Language Modelscited by 27
38 more
  1. 2025Bridging the Gap Between Ideal and Real-world Evaluation: Benchmarking AI-Generated Image Detection in Challenging Scenarioscited by 24
  2. 2024Bi-LORA: A Vision-Language Approach for Synthetic Image Detectioncited by 20
  3. 2025Toward Generalizable Forgery Detection and Reasoningcited by 20
  4. 2024DeCLIP: Decoding CLIP representations for deepfake localizationcited by 17
  5. 2025Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Modelscited by 13
  6. 2024Vulnerabilities in AI-generated Image Detection: The Challenge of Adversarial Attackscited by 12
  7. 2025FakeScope: Large Multimodal Expert Model for Transparent AI-Generated Image Forensicscited by 11
  8. 2024T2Vs Meet VLMs: A Scalable Multimodal Dataset for Visual Harmfulness Recognitioncited by 10
  9. 2025Interpretable and Reliable Detection of AI-Generated Images via Grounded Reasoning in MLLMscited by 9
  10. 2025Navigating the Challenges of AI-Generated Image Detection in the Wild: What Truly Matters?cited by 8
  11. 2025RAIDX: A Retrieval-Augmented Generation and GRPO Reinforcement Learning Framework for Explainable Deepfake Detectioncited by 8
  12. 2025DFBench: Benchmarking Deepfake Image Detection Capability of Large Multimodal Modelscited by 7
  13. 2025ThinkFake: Reasoning in Multimodal Large Language Models for AI-Generated Image Detectioncited by 7
  14. 2026Venus-DeFakerOne: Unified Fake Image Detection & Localizationcited by 7
  15. 2025Multiverse Through Deepfakes: The MultiFakeVerse Dataset of Person-Centric Visual and Conceptual Manipulationscited by 5
  16. 2025From Evidence to Verdict: An Agent-Based Forensic Framework for AI-Generated Image Detectioncited by 5
  17. 2024A Hitchhikers Guide to Fine-Grained Face Forgery Detection Using Common Sense Reasoningcited by 4
  18. 2025Locate-Then-Examine: Grounded Region Reasoning Improves Detection of AI-Generated Imagescited by 4
  19. 2025REVEAL: Reasoning-Enhanced Forensic Evidence Analysis for Explainable AI-Generated Image Detectioncited by 4
  20. 2024Present and Future Generalization of Synthetic Image Detectorscited by 3
  21. 2025LLMs Are Not Yet Ready for Deepfake Image Detectioncited by 3
  22. 2025FakeParts: a New Family of AI-Generated DeepFakescited by 3
  23. 2026UniGenDet: A Unified Generative-Discriminative Framework for Co-Evolutionary Image Generation and Generated Image Detectioncited by 3
  24. 2026ReAlign: Generalizable Image Forgery Detection via Reasoning-Aligned Representationcited by 3
  25. 2026FakeVLM-R1: Internalizing Physical Laws via CoT for Synthetic Image Detectioncited by 3
  26. 2024FIDAVL: Fake Image Detection and Attribution using Vision-Language Modelcited by 2
  27. 2025Visual Language Models as Zero-Shot Deepfake Detectorscited by 2
  28. 2025DF-LLaVA: Unlocking MLLMs for Synthetic Image Detection via Knowledge Injection and Conflict-Driven Self-Reflectioncited by 2
  29. 2026Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detectioncited by 2
  30. 2025Knowledge-Guided Prompt Learning for Deepfake Facial Image Detectioncited by 1
  31. 2025AuthGuard: Generalizable Deepfake Detection via Language Guidancecited by 1
  32. 2025CorrDetail: Visual Detail Enhanced Self-Correction for Face Forgery Detectioncited by 1
  33. 2025GAMMA: Generalizable Alignment via Multi-task and Manipulation-Augmented Training for AI-Generated Image Detectioncited by 1
  34. 2025Investigating self-supervised representations for audio-visual deepfake detectioncited by 1
  35. 2026Investigating the Viability of Employing Multi-modal Large Language Models in the Context of Audio Deepfake Detectioncited by 1
  36. 2026TranX-Adapter: Bridging Artifacts and Semantics within MLLMs for Robust AI-generated Image Detectioncited by 1
  37. 2026Rethinking VLMs for Image Forgery Detection and Localizationcited by 1
  38. 2026From Masks to Pixels and Meaning: A New Taxonomy, Benchmark, and Metrics for VLM Image Tamperingcited by 1

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Standing on the Shoulders of Giants: Reprogramming Visual-Language Model for General Deepfake Detection
  2. 2025TruthLens:A Training-Free Paradigm for DeepFake Detection
  3. 2023Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images
  4. 2024Conditioned Prompt-Optimization for Continual Deepfake Detection
  5. 2024CLIPping the Deception: Adapting Vision-Language Models for Universal Deepfake Detection
  6. 2024Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges
  7. 2025Generating and Detecting Various Types of Fake Image and Audio Content: A Review of Modern Deep Learning Technologies and Tools
  8. 2025Zero-Shot Visual Deepfake Detection: Can AI Predict and Prevent Fake Content Before It's Created?