# PVLM: Parsing-Aware Vision Language Model with Dynamic Contrastive Learning for Zero-Shot Deepfake Attribution arXiv: https://arxiv.org/abs/2504.14129 ยท HTML: https://mlsanity.com/paper/2504.14129/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. - Builds on 33 of its 50 references that are in the map. - First posted on arXiv 19 Apr 2025. ## Placed in - Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/) ## Builds on (most cited first, 30 of 33) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016) - [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018) - [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017) - [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016) - [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019) - [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020) - [Learning Confidence for Out-of-Distribution Detection in Neural Networks](https://mlsanity.com/paper/1802.04865/) (2018) - [WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection](https://mlsanity.com/paper/2101.01456/) (2021) - [DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models](https://mlsanity.com/paper/2210.06998/) (2022) - [The DeepFake Detection Challenge (DFDC) Dataset](https://mlsanity.com/paper/2006.07397/) (2020) - [DF40: Toward Next-Generation Deepfake Detection](https://mlsanity.com/paper/2406.13495/) (2024) - [C2P-CLIP: Injecting Category Common Prompt in CLIP to Enhance Generalization in Deepfake Detection](https://mlsanity.com/paper/2408.09647/) (2024) - [Diversity is Definitely Needed: Improving Model-Agnostic Zero-shot Classification via Stable Diffusion](https://mlsanity.com/paper/2302.03298/) (2023) - [Deepfake Video Detection Using Convolutional Vision Transformer](https://mlsanity.com/paper/2102.11126/) (2021) - [DiffFace: Diffusion-based Face Swapping with Facial Guidance](https://mlsanity.com/paper/2212.13344/) (2022) - [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024) - [Towards Discovery and Attribution of Open-world GAN Generated Images](https://mlsanity.com/paper/2105.04580/) (2021) - [Improved Zero-Shot Classification by Adapting VLMs with Text Descriptions](https://mlsanity.com/paper/2401.02460/) (2024) - [Deepfake Network Architecture Attribution](https://mlsanity.com/paper/2202.13843/) (2022) - [Diffusion Facial Forgery Detection](https://mlsanity.com/paper/2401.15859/) (2024) - [CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classification](https://mlsanity.com/paper/2402.17417/) (2024) - [Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models](https://mlsanity.com/paper/2409.07269/) (2024) - [Label Propagation for Zero-shot Classification with Vision-Language Models](https://mlsanity.com/paper/2404.04072/) (2024) - [Contrastive Pseudo Learning for Open-World DeepFake Attribution](https://mlsanity.com/paper/2309.11132/) (2023) - [LampMark: Proactive Deepfake Detection via Training-Free Landmark Perceptual Watermarks](https://mlsanity.com/paper/2411.17209/) (2024) - [Contrastive-center loss for deep neural networks](https://mlsanity.com/paper/1707.07391/) (2017) - [SegFace: Face Segmentation of Long-Tail Classes](https://mlsanity.com/paper/2412.08647/) (2024) - [Towards Benchmarking and Evaluating Deepfake Detection](https://mlsanity.com/paper/2203.02115/) (2022) ## Built on by (most cited first, 1 of 1) - [GazeCLIP: Gaze-Guided CLIP with Adaptive-Enhanced Fine-Grained Language Prompt for Deepfake Attribution and Detection](https://mlsanity.com/paper/2603.29295/) (2026) ## Abstract (arXiv) The challenge of tracing the source attribution of forged faces has gained significant attention due to the rapid advancement of generative models. However, existing deepfake attribution (DFA) works primarily focus on the interaction among various domains in vision modality, and other modalities such as texts and face parsing are not fully explored. Besides, they tend to fail to assess the generalization performance of deepfake attributors to unseen advanced generators like diffusion in a fine-grained manner. In this paper, we propose a novel parsing-aware vision language model with a dynamic contrastive learning (PVLM) method for zero-shot deepfake attribution (ZSDFA), which facilitates effective and fine-grained traceability to unseen advanced generators. Specifically, we conduct a novel and fine-grained ZS-DFA benchmark to evaluate the attribution performance of deepfake attributors to unseen advanced generators like diffusion. Besides, we propose an innovative PVLM attributor based on the vision-language model to capture general and diverse attribution features. We are motivated by the observation that the preservation of source face attributes in facial images generated by GAN and diffusion models varies significantly. We propose to employ the inherent facial attributes preservation differences to capture face parsing-aware forgery representations. Therefore, we devise a novel parsing encoder to focus on global face attribute embeddings, enabling parsing-guided DFA representation learning via dynamic vision-parsing matching. Additionally, we present a novel deepfake attribution contrastive center loss to pull relevant generators closer and push irrelevant ones away, which can be introduced into DFA models to enhance traceability. Experimental results show that our model exceeds the state-of-the-art on the ZS-DFA benchmark via various protocol evaluations.