# Towards Generic Image Manipulation Detection with Weakly-Supervised Self-Consistency Learning

arXiv: https://arxiv.org/abs/2309.01246 · HTML: https://mlsanity.com/paper/2309.01246/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 69 references that are in the map.
- First posted on arXiv 3 Sep 2023.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection](https://mlsanity.com/paper/2001.00179/) (2020)
- [Learning Self-Consistency for Deepfake Detection](https://mlsanity.com/paper/2012.09311/) (2020)
- [Single-Stage Semantic Segmentation from Image Labels](https://mlsanity.com/paper/2005.08104/) (2020)
- [ManiGAN: Text-Guided Image Manipulation](https://mlsanity.com/paper/1912.06203/) (2019)
- [Learning Rich Features for Image Manipulation Detection](https://mlsanity.com/paper/1805.04953/) (2018)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [UntrimmedNets for Weakly Supervised Action Recognition and Detection](https://mlsanity.com/paper/1703.03329/) (2017)
- [MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection](https://mlsanity.com/paper/2112.08935/) (2021)
- [Image Manipulation Detection by Multi-View Multi-Scale Supervision](https://mlsanity.com/paper/2104.06832/) (2021)
- [PCL: Proposal Cluster Learning for Weakly Supervised Object Detection](https://mlsanity.com/paper/1807.03342/) (2018)
- [SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization](https://mlsanity.com/paper/2009.00726/) (2020)
- [Weakly Supervised Instance Segmentation using Class Peak Response](https://mlsanity.com/paper/1804.00880/) (2018)
- [Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)](https://mlsanity.com/paper/1709.02016/) (2017)
- [Expressing Visual Relationships via Language](https://mlsanity.com/paper/1906.07689/) (2019)
- [ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization](https://mlsanity.com/paper/1609.04331/) (2016)
- [Fully Convolutional Multi-Class Multiple Instance Learning](https://mlsanity.com/paper/1412.7144/) (2014)
- [Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches](https://mlsanity.com/paper/2001.02890/) (2020)
- [Generate, Segment and Refine: Towards Generic Manipulation Segmentation](https://mlsanity.com/paper/1811.09729/) (2018)
- [A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains](https://mlsanity.com/paper/1912.11186/) (2019)
- [SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches](https://mlsanity.com/paper/2111.15078/) (2021)
- [Constrained R-CNN: A general image manipulation detection model](https://mlsanity.com/paper/1911.08217/) (2019)
- [Forensic Similarity for Digital Images](https://mlsanity.com/paper/1902.04684/) (2019)

## Built on by (most cited first, 23 of 23)

- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [LightGen: Efficient Image Generation through Knowledge Distillation and Direct Preference Optimization](https://mlsanity.com/paper/2503.08619/) (2025)
- [MH-pFLID: Model Heterogeneous personalized Federated Learning via Injection and Distillation for Medical Data Analysis](https://mlsanity.com/paper/2405.06822/) (2024)
- [Spectrum AUC Difference (SAUCD): Human-aligned 3D Shape Evaluation](https://mlsanity.com/paper/2403.01619/) (2024)
- [Tracing Hyperparameter Dependencies for Model Parsing via Learnable Graph Pooling Network](https://mlsanity.com/paper/2312.02224/) (2023)
- [StableGuard: Towards Unified Copyright Protection and Tamper Localization in Latent Diffusion Models](https://mlsanity.com/paper/2509.17993/) (2025)
- [Weakly-Supervised Image Forgery Localization via Vision-Language Collaborative Reasoning Framework](https://mlsanity.com/paper/2508.01338/) (2025)
- [Dense Feature Interaction Network for Image Inpainting Localization](https://mlsanity.com/paper/2408.02191/) (2024)
- [Beyond Fully Supervised Pixel Annotations: Scribble-Driven Weakly-Supervised Framework for Image Manipulation Localization](https://mlsanity.com/paper/2507.13018/) (2025)
- [Webly-Supervised Image Manipulation Localization via Category-Aware Auto-Annotation](https://mlsanity.com/paper/2508.20987/) (2025)
- [Divide and Fuse: Body Part Mesh Recovery from Partially Visible Human Images](https://mlsanity.com/paper/2407.09694/) (2024)
- [Context-Aware Weakly Supervised Image Manipulation Localization with SAM Refinement](https://mlsanity.com/paper/2503.20294/) (2025)
- [From Passive Perception to Active Memory: A Weakly Supervised Image Manipulation Localization Framework Driven by Coarse-Grained Annotations](https://mlsanity.com/paper/2511.20359/) (2025)
- [HRGR: Enhancing Image Manipulation Detection via Hierarchical Region-aware Graph Reasoning](https://mlsanity.com/paper/2410.21861/) (2024)
- [Learning Universal Features for Generalizable Image Forgery Localization](https://mlsanity.com/paper/2504.07462/) (2025)
- [Training-Free In-Context Forensic Chain for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2510.10111/) (2025)
- [Fine-Grained DINO Tuning with Dual Supervision for Face Forgery Detection](https://mlsanity.com/paper/2511.12107/) (2025)
- [CIEC: Coupling Implicit and Explicit Cues for Multimodal Weakly Supervised Manipulation Localization](https://mlsanity.com/paper/2602.02175/) (2026)
- [FastForensics: Efficient Two-Stream Design for Real-Time Image Manipulation Detection](https://mlsanity.com/paper/2408.16582/) (2024)
- [Weakly-supervised Localization of Manipulated Image Regions Using Multi-resolution Learned Features](https://mlsanity.com/paper/2505.23586/) (2025)
- [SAPL: Semantic-Agnostic Prompt Learning in CLIP for Weakly Supervised Image Manipulation Localization](https://mlsanity.com/paper/2601.06222/) (2026)
- [SGF-CDNet: A Consistency-Discrepancy Graph Network over Semantic-Geometric Fused Nodes for Face Forgery Detection](https://mlsanity.com/paper/2607.03883/) (2026)
- [Less Supervision, Better Generalization: Weakly Supervised Fake Region Localization in Diffusion-Edited Images](https://mlsanity.com/paper/2609.36882/) (2026)

## Abstract (arXiv)

As advanced image manipulation techniques emerge, detecting the manipulation becomes increasingly important. Despite the success of recent learning-based approaches for image manipulation detection, they typically require expensive pixel-level annotations to train, while exhibiting degraded performance when testing on images that are differently manipulated compared with training images. To address these limitations, we propose weakly-supervised image manipulation detection, such that only binary image-level labels (authentic or tampered with) are required for training purpose. Such a weakly-supervised setting can leverage more training images and has the potential to adapt quickly to new manipulation techniques. To improve the generalization ability, we propose weakly-supervised self-consistency learning (WSCL) to leverage the weakly annotated images. Specifically, two consistency properties are learned: multi-source consistency (MSC) and inter-patch consistency (IPC). MSC exploits different content-agnostic information and enables cross-source learning via an online pseudo label generation and refinement process. IPC performs global pair-wise patch-patch relationship reasoning to discover a complete region of manipulation. Extensive experiments validate that our WSCL, even though is weakly supervised, exhibits competitive performance compared with fully-supervised counterpart under both in-distribution and out-of-distribution evaluations, as well as reasonable manipulation localization ability.