# Rethinking Image Forgery Detection via Soft Contrastive Learning and Unsupervised Clustering

arXiv: https://arxiv.org/abs/2308.09307 · HTML: https://mlsanity.com/paper/2308.09307/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 73 references that are in the map.
- First posted on arXiv 18 Aug 2023.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [Decoupled Contrastive Learning](https://mlsanity.com/paper/2110.06848/) (2021)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization](https://mlsanity.com/paper/2212.10957/) (2022)
- [MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection](https://mlsanity.com/paper/2112.08935/) (2021)
- [Noiseprint: a CNN-based camera model fingerprint](https://mlsanity.com/paper/1808.08396/) (2018)
- [Dual Contrastive Learning for General Face Forgery Detection](https://mlsanity.com/paper/2112.13522/) (2021)
- [Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2108.12947/) (2021)
- [Deep k-Means: Jointly clustering with k-Means and learning representations](https://mlsanity.com/paper/1806.10069/) (2018)
- [Towards Benchmarking and Evaluating Deepfake Detection](https://mlsanity.com/paper/2203.02115/) (2022)
- [CFL-Net: Image Forgery Localization Using Contrastive Learning](https://mlsanity.com/paper/2210.02182/) (2022)
- [Towards Effective Image Manipulation Detection with Proposal Contrastive Learning](https://mlsanity.com/paper/2210.08529/) (2022)
- [Diff-ID: An Explainable Identity Difference Quantification Framework for DeepFake Detection](https://mlsanity.com/paper/2303.18174/) (2023)
- [Image Splicing Detection, Localization and Attribution via JPEG Primary Quantization Matrix Estimation and Clustering](https://mlsanity.com/paper/2102.01439/) (2021)

## Built on by (most cited first, 12 of 12)

- [SAFIRE: Segment Any Forged Image Region](https://mlsanity.com/paper/2412.08197/) (2024)
- [Exploring Multi-view Pixel Contrast for General and Robust Image Forgery Localization](https://mlsanity.com/paper/2406.13565/) (2024)
- [AgentFoX: LLM Agent-Guided Fusion with eXplainability for AI-Generated Image Detection](https://mlsanity.com/paper/2603.23115/) (2026)
- [Learning Universal Features for Generalizable Image Forgery Localization](https://mlsanity.com/paper/2504.07462/) (2025)
- [FeatDistill: A Feature Distillation Enhanced Multi-Expert Ensemble Framework for Robust AI-generated Image Detection](https://mlsanity.com/paper/2603.21939/) (2026)
- [Semantic Manipulation Localization](https://mlsanity.com/paper/2604.10132/) (2026)
- [Trusted Video Inpainting Localization via Deep Attentive Noise Learning](https://mlsanity.com/paper/2406.13576/) (2024)
- [Wavelet based inpainting detection](https://mlsanity.com/paper/2408.06429/) (2024)
- [Enhanced Wavelet Scattering Network for image inpainting detection](https://mlsanity.com/paper/2409.17023/) (2024)
- [RelayFormer: A Unified Local-Global Attention Framework for Scalable Image and Video Manipulation Localization](https://mlsanity.com/paper/2508.09459/) (2025)
- [PromptForge-350k: A Large-Scale Dataset and Contrastive Framework for Prompt-Based AI Image Forgery Localization](https://mlsanity.com/paper/2603.29386/) (2026)
- [When Modalities Fail to Tango: Conformal Backdoor Detection in Multimodal Contrastive Learning](https://mlsanity.com/paper/2608.04052/) (2026)

## Abstract (arXiv)

Image forgery detection aims to detect and locate forged regions in an image. Most existing forgery detection algorithms formulate classification problems to classify pixels into forged or pristine. However, the definition of forged and pristine pixels is only relative within one single image, e.g., a forged region in image A is actually a pristine one in its source image B (splicing forgery). Such a relative definition has been severely overlooked by existing methods, which unnecessarily mix forged (pristine) regions across different images into the same category. To resolve this dilemma, we propose the FOrensic ContrAstive cLustering (FOCAL) method, a novel, simple yet very effective paradigm based on soft contrastive learning and unsupervised clustering for the image forgery detection. Specifically, FOCAL 1) designs a soft contrastive learning (SCL) to supervise the high-level forensic feature extraction in an image-by-image manner, explicitly reflecting the above relative definition; 2) employs an on-the-fly unsupervised clustering algorithm (instead of a trained one) to cluster the learned features into forged/pristine categories, further suppressing the cross-image influence from training data; and 3) allows to further boost the detection performance via simple feature-level concatenation without the need of retraining. Extensive experimental results over six public testing datasets demonstrate that our proposed FOCAL significantly outperforms the state-of-the-art competitors by big margins: +24.8% on Coverage, +18.9% on Columbia, +17.3% on FF++, +15.3% on MISD, +15.0% on CASIA and +10.5% on NIST in terms of IoU (see also Fig. 1). The paradigm of FOCAL could bring fresh insights and serve as a novel benchmark for the image forgery detection task. The code is available at https://github.com/HighwayWu/FOCAL.