# IML-ViT: Benchmarking Image Manipulation Localization by Vision Transformer

arXiv: https://arxiv.org/abs/2307.14863 · HTML: https://mlsanity.com/paper/2307.14863/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 27 Jul 2023.

## Builds on (most cited first, 26 of 26)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [Media Forensics and DeepFakes: an overview](https://mlsanity.com/paper/2001.06564/) (2020)
- [Learning Rich Features for Image Manipulation Detection](https://mlsanity.com/paper/1805.04953/) (2018)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [Hierarchical Fine-Grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2303.17111/) (2023)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [Benchmarking Detection Transfer Learning with Vision Transformers](https://mlsanity.com/paper/2111.11429/) (2021)
- [TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization](https://mlsanity.com/paper/2212.10957/) (2022)
- [MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection](https://mlsanity.com/paper/2112.08935/) (2021)
- [Image Manipulation Detection by Multi-View Multi-Scale Supervision](https://mlsanity.com/paper/2104.06832/) (2021)
- [SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization](https://mlsanity.com/paper/2009.00726/) (2020)
- [Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2108.12947/) (2021)
- [ObjectFormer for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2203.14681/) (2022)
- [Generate, Segment and Refine: Towards Generic Manipulation Segmentation](https://mlsanity.com/paper/1811.09729/) (2018)
- [Constrained R-CNN: A general image manipulation detection model](https://mlsanity.com/paper/1911.08217/) (2019)
- [TransForensics: Image Forgery Localization with Dense Self-Attention](https://mlsanity.com/paper/2108.03871/) (2021)
- [CFL-Net: Image Forgery Localization Using Contrastive Learning](https://mlsanity.com/paper/2210.02182/) (2022)

## Built on by (most cited first, 30 of 44)

- [Can We Get Rid of Handcrafted Feature Extractors? SparseViT: Nonsemantics-Centered, Parameter-Efficient Image Manipulation Localization through Spare-Coding Transformer](https://mlsanity.com/paper/2412.14598/) (2024)
- [Pre-training-free Image Manipulation Localization through Non-Mutually Exclusive Contrastive Learning](https://mlsanity.com/paper/2309.14900/) (2023)
- [GIM: A Million-scale Benchmark for Generative Image Manipulation Detection and Localization](https://mlsanity.com/paper/2406.16531/) (2024)
- [Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline](https://mlsanity.com/paper/2412.19685/) (2024)
- [AIForge-Doc: A Benchmark for Detecting AI-Forged Tampering in Financial and Form Documents](https://mlsanity.com/paper/2602.20569/) (2026)
- [Reinforced Multi-teacher Knowledge Distillation for Efficient General Image Forgery Detection and Localization](https://mlsanity.com/paper/2504.05224/) (2025)
- [FakeParts: a New Family of AI-Generated DeepFakes](https://mlsanity.com/paper/2508.21052/) (2025)
- [GS-Checker: Tampering Localization for 3D Gaussian Splatting](https://mlsanity.com/paper/2511.20354/) (2025)
- [When the Forger Is the Judge: GPT-Image-2 Cannot Recognize Its Own Faked Documents](https://mlsanity.com/paper/2604.25213/) (2026)
- [ReAlign: Generalizable Image Forgery Detection via Reasoning-Aligned Representation](https://mlsanity.com/paper/2605.16080/) (2026)
- [Leveraging Failed Samples: A Few-Shot and Training-Free Framework for Generalized Deepfake Detection](https://mlsanity.com/paper/2508.09475/) (2025)
- [A Noise and Edge extraction-based dual-branch method for Shallowfake and Deepfake Localization](https://mlsanity.com/paper/2409.00896/) (2024)
- [Learning Universal Features for Generalizable Image Forgery Localization](https://mlsanity.com/paper/2504.07462/) (2025)
- [Bridging Semantic Logic Gaps: A Cognition Inspired Multimodal Boundary Preserving Network for Image Manipulation Localization](https://mlsanity.com/paper/2508.07216/) (2025)
- [Detecting Text Manipulation in Images using Vision Language Models](https://mlsanity.com/paper/2509.10278/) (2025)
- [Rethinking VLMs for Image Forgery Detection and Localization](https://mlsanity.com/paper/2603.12930/) (2026)
- [SurFITR: A Dataset for Surveillance Image Forgery Detection and Localisation](https://mlsanity.com/paper/2604.07101/) (2026)
- [Semantic Manipulation Localization](https://mlsanity.com/paper/2604.10132/) (2026)
- [TAP into the Patch Tokens: Leveraging Vision Foundation Model Features for AI-Generated Image Detection](https://mlsanity.com/paper/2604.26772/) (2026)
- [From Forgeries to Foundation Models: A Systematic Survey of Identity Document Attack and Detection](https://mlsanity.com/paper/2607.01442/) (2026)
- [Perceptual MAE for Image Manipulation Localization: A High-level Vision Learner Focusing on Low-level Features](https://mlsanity.com/paper/2310.06525/) (2023)
- [COCO-Inpaint: A Benchmark for Detecting and Localizing Inpainting-Based Image Manipulations](https://mlsanity.com/paper/2504.18361/) (2025)
- [Unmasking Synthetic Realities in Generative AI: A Comprehensive Review of Adversarially Robust Deepfake Detection Systems](https://mlsanity.com/paper/2507.21157/) (2025)
- [EfficientIML: Efficient High-Resolution Image Manipulation Localization](https://mlsanity.com/paper/2509.08583/) (2025)
- [Unlocking the Forgery Detection Potential of Vanilla MLLMs: A Novel Training-Free Pipeline](https://mlsanity.com/paper/2511.13442/) (2025)
- [OmniFD: A Unified Model for Versatile Face Forgery Detection](https://mlsanity.com/paper/2512.01128/) (2025)
- [VAAS: Vision-Attention Anomaly Scoring for Image Manipulation Detection in Digital Forensics](https://mlsanity.com/paper/2512.15512/) (2025)
- [SAPL: Semantic-Agnostic Prompt Learning in CLIP for Weakly Supervised Image Manipulation Localization](https://mlsanity.com/paper/2601.06222/) (2026)
- [DiffFace-Edit: A Diffusion-Based Facial Dataset for Forgery-Semantic Driven Deepfake Detection Analysis](https://mlsanity.com/paper/2601.13551/) (2026)
- [Off-the-shelf Vision Models Benefit Image Manipulation Localization](https://mlsanity.com/paper/2604.09096/) (2026)

## Abstract (arXiv)

Advanced image tampering techniques are increasingly challenging the trustworthiness of multimedia, leading to the development of Image Manipulation Localization (IML). But what makes a good IML model? The answer lies in the way to capture artifacts. Exploiting artifacts requires the model to extract non-semantic discrepancies between manipulated and authentic regions, necessitating explicit comparisons between the two areas. With the self-attention mechanism, naturally, the Transformer should be a better candidate to capture artifacts. However, due to limited datasets, there is currently no pure ViT-based approach for IML to serve as a benchmark, and CNNs dominate the entire task. Nevertheless, CNNs suffer from weak long-range and non-semantic modeling. To bridge this gap, based on the fact that artifacts are sensitive to image resolution, amplified under multi-scale features, and massive at the manipulation border, we formulate the answer to the former question as building a ViT with high-resolution capacity, multi-scale feature extraction capability, and manipulation edge supervision that could converge with a small amount of data. We term this simple but effective ViT paradigm IML-ViT, which has significant potential to become a new benchmark for IML. Extensive experiments on three different mainstream protocols verified our model outperforms the state-of-the-art manipulation localization methods. Code and models are available at https://github.com/SunnyHaze/IML-ViT.