# Pixel-Inconsistency Modeling for Image Manipulation Localization

arXiv: https://arxiv.org/abs/2310.00234 · HTML: https://mlsanity.com/paper/2310.00234/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 112 references that are in the map.
- First posted on arXiv 30 Sep 2023.

## Builds on (most cited first, 30 of 47)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications](https://mlsanity.com/paper/1701.05517/) (2017)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Media Forensics and DeepFakes: an overview](https://mlsanity.com/paper/2001.06564/) (2020)
- [Searching Central Difference Convolutional Networks for Face Anti-Spoofing](https://mlsanity.com/paper/2003.04092/) (2020)
- [Pixel Difference Networks for Efficient Edge Detection](https://mlsanity.com/paper/2108.07009/) (2021)
- [EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning](https://mlsanity.com/paper/1901.00212/) (2019)
- [Learning Rich Features for Image Manipulation Detection](https://mlsanity.com/paper/1805.04953/) (2018)
- [PixelSNAIL: An Improved Autoregressive Generative Model](https://mlsanity.com/paper/1712.09763/) (2017)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization](https://mlsanity.com/paper/2212.10957/) (2022)
- [MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection](https://mlsanity.com/paper/2112.08935/) (2021)
- [Noiseprint: a CNN-based camera model fingerprint](https://mlsanity.com/paper/1808.08396/) (2018)
- [Explicit Visual Prompting for Low-Level Structure Segmentations](https://mlsanity.com/paper/2303.10883/) (2023)

## Built on by (most cited first, 30 of 33)

- [Transferable Adversarial Attacks on SAM and Its Downstream Models](https://mlsanity.com/paper/2410.20197/) (2024)
- [GIM: A Million-scale Benchmark for Generative Image Manipulation Detection and Localization](https://mlsanity.com/paper/2406.16531/) (2024)
- [Exploring Multi-view Pixel Contrast for General and Robust Image Forgery Localization](https://mlsanity.com/paper/2406.13565/) (2024)
- [Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems](https://mlsanity.com/paper/2503.00383/) (2025)
- [Weakly-supervised Audio Temporal Forgery Localization via Progressive Audio-language Co-learning Network](https://mlsanity.com/paper/2505.01880/) (2025)
- [ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack](https://mlsanity.com/paper/2508.07402/) (2025)
- [Progressive Feedback-Enhanced Transformer for Image Forgery Localization](https://mlsanity.com/paper/2311.08910/) (2023)
- [FractalForensics: Proactive Deepfake Detection and Localization via Fractal Watermarks](https://mlsanity.com/paper/2504.09451/) (2025)
- [PROMPT-IML: Image Manipulation Localization with Pre-trained Foundation Models Through Prompt Tuning](https://mlsanity.com/paper/2401.00653/) (2024)
- [Dense Feature Interaction Network for Image Inpainting Localization](https://mlsanity.com/paper/2408.02191/) (2024)
- [Pleno-Generation: A Scalable Generative Face Video Compression Framework with Bandwidth Intelligence](https://mlsanity.com/paper/2502.17085/) (2025)
- [Generative Models at the Frontier of Compression: A Survey on Generative Face Video Coding](https://mlsanity.com/paper/2506.07369/) (2025)
- [Beyond Fully Supervised Pixel Annotations: Scribble-Driven Weakly-Supervised Framework for Image Manipulation Localization](https://mlsanity.com/paper/2507.13018/) (2025)
- [Webly-Supervised Image Manipulation Localization via Category-Aware Auto-Annotation](https://mlsanity.com/paper/2508.20987/) (2025)
- [DDA-Thinker: Decoupled Dual-Atomic Reinforcement Learning for Reasoning-Driven Image Editing](https://mlsanity.com/paper/2604.25477/) (2026)
- [Image Forgery Localization with State Space Models](https://mlsanity.com/paper/2412.11214/) (2024)
- [Propose and Rectify: A Forensics-Driven MLLM Framework for Image Manipulation Localization](https://mlsanity.com/paper/2508.17976/) (2025)
- [From Passive Perception to Active Memory: A Weakly Supervised Image Manipulation Localization Framework Driven by Coarse-Grained Annotations](https://mlsanity.com/paper/2511.20359/) (2025)
- [Aggregating Diverse Cue Experts for AI-Generated Image Detection](https://mlsanity.com/paper/2601.08790/) (2026)
- [ForgeryVCR: Visual-Centric Reasoning via Efficient Forensic Tools in MLLMs for Image Forgery Detection and Localization](https://mlsanity.com/paper/2602.14098/) (2026)
- [Bridging Semantic Logic Gaps: A Cognition Inspired Multimodal Boundary Preserving Network for Image Manipulation Localization](https://mlsanity.com/paper/2508.07216/) (2025)
- [CLUE: Leveraging Low-Rank Adaptation to Capture Latent Uncovered Evidence for Image Forgery Localization](https://mlsanity.com/paper/2508.07413/) (2025)
- [FeatDistill: A Feature Distillation Enhanced Multi-Expert Ensemble Framework for Robust AI-generated Image Detection](https://mlsanity.com/paper/2603.21939/) (2026)
- [Towards General Visual-Linguistic Face Forgery Detection(V2)](https://mlsanity.com/paper/2502.20698/) (2025)
- [VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models: Methods and Results](https://mlsanity.com/paper/2509.09190/) (2025)
- [SAPL: Semantic-Agnostic Prompt Learning in CLIP for Weakly Supervised Image Manipulation Localization](https://mlsanity.com/paper/2601.06222/) (2026)
- [Towards Generalizable Deepfake Detection via Real Distribution Bias Correction](https://mlsanity.com/paper/2603.14005/) (2026)
- [The Courtroom Trial of Pixels: Robust Image Manipulation Localization via Adversarial Evidence and Reinforcement Learning Judgment](https://mlsanity.com/paper/2604.14703/) (2026)
- [Whether, Which, and Whose: Solving the Triple Challenge of Deepfake Proactive Forensics in Multi-Face Scenarios](https://mlsanity.com/paper/2604.26342/) (2026)
- [Towards Generalized Image Manipulation Localization via Score-based Model](https://mlsanity.com/paper/2605.16879/) (2026)

## Abstract (arXiv)

Digital image forensics plays a crucial role in image authentication and manipulation localization. Despite the progress powered by deep neural networks, existing forgery localization methodologies exhibit limitations when deployed to unseen datasets and perturbed images (i.e., lack of generalization and robustness to real-world applications). To circumvent these problems and aid image integrity, this paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. The rationale is grounded on the observation that most image signal processors (ISP) involve the demosaicing process, which introduces pixel correlations in pristine images. Moreover, manipulating operations, including splicing, copy-move, and inpainting, directly affect such pixel regularity. We, therefore, first split the input image into several blocks and design masked self-attention mechanisms to model the global pixel dependency in input images. Simultaneously, we optimize another local pixel dependency stream to mine local manipulation clues within input forgery images. In addition, we design novel Learning-to-Weight Modules (LWM) to combine features from the two streams, thereby enhancing the final forgery localization performance. To improve the training process, we propose a novel Pixel-Inconsistency Data Augmentation (PIDA) strategy, driving the model to focus on capturing inherent pixel-level artifacts instead of mining semantic forgery traces. This work establishes a comprehensive benchmark integrating 15 representative detection models across 12 datasets. Extensive experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints and achieve state-of-the-art generalization and robustness performances in image manipulation localization.