# AdaptPrompt: Parameter-Efficient Adaptation of VLMs for Generalizable Deepfake Detection

arXiv: https://arxiv.org/abs/2512.17730 · HTML: https://mlsanity.com/paper/2512.17730/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 19 Dec 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020)
- [Towards Universal Fake Image Detectors that Generalize Across Generative Models](https://mlsanity.com/paper/2302.10174/) (2023)
- [Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions](https://mlsanity.com/paper/2003.01826/) (2020)
- [Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection](https://mlsanity.com/paper/2203.12208/) (2022)
- [Raising the Bar of AI-generated Image Detection with CLIP](https://mlsanity.com/paper/2312.00195/) (2023)
- [Are GAN generated images easy to detect? A critical analysis of the state-of-the-art](https://mlsanity.com/paper/2104.02617/) (2021)
- [Face Forgery Detection by 3D Decomposition](https://mlsanity.com/paper/2011.09737/) (2020)
- [CLIPping the Deception: Adapting Vision-Language Models for Universal Deepfake Detection](https://mlsanity.com/paper/2402.12927/) (2024)
- [Lighting (In)consistency of Paint by Text](https://mlsanity.com/paper/2207.13744/) (2022)
- [Perspective (In)consistency of Paint by Text](https://mlsanity.com/paper/2206.14617/) (2022)
- [FLARE up your data: Diffusion-based Augmentation Method in Astronomical Imaging](https://mlsanity.com/paper/2405.13267/) (2024)
- [CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical Imaging](https://mlsanity.com/paper/2407.07315/) (2024)
- [Introducing SDICE: An Index for Assessing Diversity of Synthetic Medical Datasets](https://mlsanity.com/paper/2409.19436/) (2024)
- [Robust and Calibrated Detection of Authentic Multimedia Content](https://mlsanity.com/paper/2512.15182/) (2025)
- [AstroSpy: On detecting Fake Images in Astronomy via Joint Image-Spectral Representations](https://mlsanity.com/paper/2407.06817/) (2024)
- [FaceAnonyMixer: Cancelable Faces via Identity Consistent Latent Space Mixing](https://mlsanity.com/paper/2508.05636/) (2025)

## Abstract (arXiv)

Detectors of AI-generated images tend to inherit the biases of the data they are trained on: models fitted to GAN imagery learn to treat GAN-specific artifacts as the very definition of "fake" and consequently miss images produced by diffusion models and commercial generation tools. We study this generalization problem from two directions. First, we introduce Diff-Gen, a balanced corpus consisting of 100k diffusion-generated samples and an equally sized set of real images, with the real subset selected to mirror the class distribution of the synthetic data. A spectral analysis shows that, unlike GAN data, Diff-Gen exhibits broad, non-periodic high-frequency energy, and we find that detectors trained on it transfer substantially better to unseen generator families. Second, we propose AdaptPrompt, a parameter-efficient adaptation of CLIP that combines a visual adapter with learnable text prompts and trains roughly 0.1% of the model's parameters. We further observe that truncating the last transformer block of the vision encoder consistently improves detection, suggesting that the final semantic-alignment layers of CLIP suppress the low-level traces on which forensic decisions rely. Across a benchmark of 25 test sets covering GANs, diffusion models, and commercial tools such as Midjourney and DALL-E 3, AdaptPrompt trained on Diff-Gen attains the best mean average precision (98.60%) and accuracy (92.72%), while matching fully fine-tuned baselines at a fraction of their training cost. We also show that the same framework supports data-efficient training and closed-set source attribution across 22 generators.