# Pre-training-free Image Manipulation Localization through Non-Mutually Exclusive Contrastive Learning

arXiv: https://arxiv.org/abs/2309.14900 · HTML: https://mlsanity.com/paper/2309.14900/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 26 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2011.10043/) (2020)
- [What makes fake images detectable? Understanding properties that generalize](https://mlsanity.com/paper/2008.10588/) (2020)
- [Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning](https://mlsanity.com/paper/2004.11660/) (2020)
- [Learning Rich Features for Image Manipulation Detection](https://mlsanity.com/paper/1805.04953/) (2018)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection](https://mlsanity.com/paper/2112.08935/) (2021)
- [Image Manipulation Detection by Multi-View Multi-Scale Supervision](https://mlsanity.com/paper/2104.06832/) (2021)
- [SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization](https://mlsanity.com/paper/2009.00726/) (2020)
- [ObjectFormer for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2203.14681/) (2022)
- [Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)](https://mlsanity.com/paper/1709.02016/) (2017)
- [IML-ViT: Benchmarking Image Manipulation Localization by Vision Transformer](https://mlsanity.com/paper/2307.14863/) (2023)
- [Forensic Similarity for Digital Images](https://mlsanity.com/paper/1902.04684/) (2019)
- [Deep Matching and Validation Network -- An End-to-End Solution to Constrained Image Splicing Localization and Detection](https://mlsanity.com/paper/1705.09765/) (2017)
- [Self-Adversarial Training incorporating Forgery Attention for Image Forgery Localization](https://mlsanity.com/paper/2107.02434/) (2021)
- [Personal Privacy Protection via Irrelevant Faces Tracking and Pixelation in Video Live Streaming](https://mlsanity.com/paper/2101.01060/) (2021)
- [Defocus Blur Detection via Depth Distillation](https://mlsanity.com/paper/2007.08113/) (2020)
- [Privacy-sensitive Objects Pixelation for Live Video Streaming](https://mlsanity.com/paper/2101.00604/) (2021)
- [TBNet:Two-Stream Boundary-aware Network for Generic Image Manipulation Localization](https://mlsanity.com/paper/2108.04508/) (2021)

## Built on by (most cited first, 23 of 23)

- [IMDL-BenCo: A Comprehensive Benchmark and Codebase for Image Manipulation Detection & Localization](https://mlsanity.com/paper/2406.10580/) (2024)
- [Mesoscopic Insights: Orchestrating Multi-scale & Hybrid Architecture for Image Manipulation Localization](https://mlsanity.com/paper/2412.13753/) (2024)
- [ForensicHub: A Unified Benchmark & Codebase for All-Domain Fake Image Detection and Localization](https://mlsanity.com/paper/2505.11003/) (2025)
- [Language-guided Hierarchical Fine-grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2410.23556/) (2024)
- [Tracing Hyperparameter Dependencies for Model Parsing via Learnable Graph Pooling Network](https://mlsanity.com/paper/2312.02224/) (2023)
- [Venus-DeFakerOne: Unified Fake Image Detection & Localization](https://mlsanity.com/paper/2605.14091/) (2026)
- [Mixture-of-Noises Enhanced Forgery-Aware Predictor for Multi-Face Manipulation Detection and Localization](https://mlsanity.com/paper/2408.02306/) (2024)
- [StableGuard: Towards Unified Copyright Protection and Tamper Localization in Latent Diffusion Models](https://mlsanity.com/paper/2509.17993/) (2025)
- [AnimeDL-2M: Million-Scale AI-Generated Anime Image Detection and Localization in Diffusion Era](https://mlsanity.com/paper/2504.11015/) (2025)
- [Webly-Supervised Image Manipulation Localization via Category-Aware Auto-Annotation](https://mlsanity.com/paper/2508.20987/) (2025)
- [Can We Build a Monolithic Model for Fake Image Detection? SICA: Semantic-Induced Constrained Adaptation for Unified-Yet-Discriminative Artifact Feature Space Reconstruction](https://mlsanity.com/paper/2602.06676/) (2026)
- [Video Inpainting Localization with Contrastive Learning](https://mlsanity.com/paper/2406.17628/) (2024)
- [HRGR: Enhancing Image Manipulation Detection via Hierarchical Region-aware Graph Reasoning](https://mlsanity.com/paper/2410.21861/) (2024)
- [NeXT-IMDL: Build Benchmark for NeXT-Generation Image Manipulation Detection & Localization](https://mlsanity.com/paper/2512.23374/) (2025)
- [M^3:Manipulation Mask Manufacturer for Arbitrary-Scale Super-Resolution Mask](https://mlsanity.com/paper/2407.03695/) (2024)
- [FastForensics: Efficient Two-Stream Design for Real-Time Image Manipulation Detection](https://mlsanity.com/paper/2408.16582/) (2024)
- [Unmasking Synthetic Realities in Generative AI: A Comprehensive Review of Adversarially Robust Deepfake Detection Systems](https://mlsanity.com/paper/2507.21157/) (2025)
- [RelayFormer: A Unified Local-Global Attention Framework for Scalable Image and Video Manipulation Localization](https://mlsanity.com/paper/2508.09459/) (2025)
- [High-Fidelity Face Content Recovery via Tamper-Resilient Versatile Watermarking](https://mlsanity.com/paper/2603.23940/) (2026)
- [PromptForge-350k: A Large-Scale Dataset and Contrastive Framework for Prompt-Based AI Image Forgery Localization](https://mlsanity.com/paper/2603.29386/) (2026)
- [The Courtroom Trial of Pixels: Robust Image Manipulation Localization via Adversarial Evidence and Reinforcement Learning Judgment](https://mlsanity.com/paper/2604.14703/) (2026)
- [DINOv3 Beats Specialized Detectors: A Simple Foundation Model Baseline for Image Forensics](https://mlsanity.com/paper/2604.16083/) (2026)
- [Towards Generalized Image Manipulation Localization via Score-based Model](https://mlsanity.com/paper/2605.16879/) (2026)

## Abstract (arXiv)

Deep Image Manipulation Localization (IML) models suffer from training data insufficiency and thus heavily rely on pre-training. We argue that contrastive learning is more suitable to tackle the data insufficiency problem for IML. Crafting mutually exclusive positives and negatives is the prerequisite for contrastive learning. However, when adopting contrastive learning in IML, we encounter three categories of image patches: tampered, authentic, and contour patches. Tampered and authentic patches are naturally mutually exclusive, but contour patches containing both tampered and authentic pixels are non-mutually exclusive to them. Simply abnegating these contour patches results in a drastic performance loss since contour patches are decisive to the learning outcomes. Hence, we propose the Non-mutually exclusive Contrastive Learning (NCL) framework to rescue conventional contrastive learning from the above dilemma. In NCL, to cope with the non-mutually exclusivity, we first establish a pivot structure with dual branches to constantly switch the role of contour patches between positives and negatives while training. Then, we devise a pivot-consistent loss to avoid spatial corruption caused by the role-switching process. In this manner, NCL both inherits the self-supervised merits to address the data insufficiency and retains a high manipulation localization accuracy. Extensive experiments verify that our NCL achieves state-of-the-art performance on all five benchmarks without any pre-training and is more robust on unseen real-life samples. The code is available at: https://github.com/Knightzjz/NCL-IML.