# Detect Any Deepfakes: Segment Anything Meets Face Forgery Detection and Localization

arXiv: https://arxiv.org/abs/2306.17075 · HTML: https://mlsanity.com/paper/2306.17075/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://mlsanity.com/paper/2110.07602/) (2021)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [DeepFakes: a New Threat to Face Recognition? Assessment and Detection](https://mlsanity.com/paper/1812.08685/) (2018)
- [Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos](https://mlsanity.com/paper/1810.11215/) (2018)
- [Hierarchical Fine-Grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2303.17111/) (2023)
- [Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos](https://mlsanity.com/paper/1906.06876/) (2019)
- [FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces](https://mlsanity.com/paper/1803.09179/) (2018)
- [A Comprehensive Survey on Segment Anything Model for Vision and Beyond](https://mlsanity.com/paper/2305.08196/) (2023)
- [FakeLocator: Robust Localization of GAN-Based Face Manipulations](https://mlsanity.com/paper/2001.09598/) (2020)
- [Detect and Locate: Exposing Face Manipulation by Semantic- and Noise-level Telltales](https://mlsanity.com/paper/2107.05821/) (2021)
- [Learnable Ophthalmology SAM](https://mlsanity.com/paper/2304.13425/) (2023)
- [Low-Rank Modeling and Its Applications in Image Analysis](https://mlsanity.com/paper/1401.3409/) (2014)
- [Collaborative Feature Learning for Fine-grained Facial Forgery Detection and Segmentation](https://mlsanity.com/paper/2304.08078/) (2023)

## Built on by (most cited first, 20 of 20)

- [SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models](https://mlsanity.com/paper/2402.04178/) (2024)
- [Credit Card Fraud Detection Using Advanced Transformer Model](https://mlsanity.com/paper/2406.03733/) (2024)
- [Deep Learning-based Anomaly Detection and Log Analysis for Computer Networks](https://mlsanity.com/paper/2407.05639/) (2024)
- [GM-DF: Generalized Multi-Scenario Deepfake Detection](https://mlsanity.com/paper/2406.20078/) (2024)
- [Evaluating Modern Approaches in 3D Scene Reconstruction: NeRF vs Gaussian-Based Methods](https://mlsanity.com/paper/2408.04268/) (2024)
- [MAS-SAM: Segment Any Marine Animal with Aggregated Features](https://mlsanity.com/paper/2404.15700/) (2024)
- [Delocate: Detection and Localization for Deepfake Videos with Randomly-Located Tampered Traces](https://mlsanity.com/paper/2401.13516/) (2024)
- [Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook](https://mlsanity.com/paper/2411.19537/) (2024)
- [Mapping New Realities: Ground Truth Image Creation with Pix2Pix Image-to-Image Translation](https://mlsanity.com/paper/2404.19265/) (2024)
- [Segment Anything for Videos: A Systematic Survey](https://mlsanity.com/paper/2408.08315/) (2024)
- [Evaluating SAM2 for Video Semantic Segmentation](https://mlsanity.com/paper/2512.01774/) (2025)
- [A Noise and Edge extraction-based dual-branch method for Shallowfake and Deepfake Localization](https://mlsanity.com/paper/2409.00896/) (2024)
- [Frequency Domain Enhanced U-Net for Low-Frequency Information-Rich Image Segmentation in Surgical and Deep-Sea Exploration Robots](https://mlsanity.com/paper/2502.03829/) (2025)
- [Patch-Discontinuity Mining for Generalized Deepfake Detection](https://mlsanity.com/paper/2512.22027/) (2025)
- [Detection of Synthetic Face Images: Accuracy, Robustness, Generalization](https://mlsanity.com/paper/2406.17547/) (2024)
- [LSP-ST: Ladder Shape-Biased Side-Tuning for Robust Infrared Small Target Detection](https://mlsanity.com/paper/2504.14481/) (2025)
- [Unmasking Synthetic Realities in Generative AI: A Comprehensive Review of Adversarially Robust Deepfake Detection Systems](https://mlsanity.com/paper/2507.21157/) (2025)
- [OmniFD: A Unified Model for Versatile Face Forgery Detection](https://mlsanity.com/paper/2512.01128/) (2025)
- [HFP-SAM: Hierarchical Frequency Prompted SAM for Efficient Marine Animal Segmentation](https://mlsanity.com/paper/2603.12708/) (2026)
- [MFVLR: Multi-domain Fine-grained Vision-Language Reconstruction for Generalizable Diffusion Face Forgery Detection and Localization](https://mlsanity.com/paper/2605.10071/) (2026)

## Abstract (arXiv)

The rapid advancements in computer vision have stimulated remarkable progress in face forgery techniques, capturing the dedicated attention of researchers committed to detecting forgeries and precisely localizing manipulated areas. Nonetheless, with limited fine-grained pixel-wise supervision labels, deepfake detection models perform unsatisfactorily on precise forgery detection and localization. To address this challenge, we introduce the well-trained vision segmentation foundation model, i.e., Segment Anything Model (SAM) in face forgery detection and localization. Based on SAM, we propose the Detect Any Deepfakes (DADF) framework with the Multiscale Adapter, which can capture short- and long-range forgery contexts for efficient fine-tuning. Moreover, to better identify forged traces and augment the model's sensitivity towards forgery regions, Reconstruction Guided Attention (RGA) module is proposed. The proposed framework seamlessly integrates end-to-end forgery localization and detection optimization. Extensive experiments on three benchmark datasets demonstrate the superiority of our approach for both forgery detection and localization. The codes will be released soon at https://github.com/laiyingxin2/DADF.