# Context-Aware Weakly Supervised Image Manipulation Localization with SAM Refinement

arXiv: https://arxiv.org/abs/2503.20294 · HTML: https://mlsanity.com/paper/2503.20294/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 26 Mar 2025.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Single-Stage Semantic Segmentation from Image Labels](https://mlsanity.com/paper/2005.08104/) (2020)
- [Conformer: Local Features Coupling Global Representations for Visual Recognition](https://mlsanity.com/paper/2105.03889/) (2021)
- [MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection](https://mlsanity.com/paper/2112.08935/) (2021)
- [Image Manipulation Detection by Multi-View Multi-Scale Supervision](https://mlsanity.com/paper/2104.06832/) (2021)
- [ObjectFormer for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2203.14681/) (2022)
- [Fully Convolutional Multi-Class Multiple Instance Learning](https://mlsanity.com/paper/1412.7144/) (2014)
- [Mesoscopic Insights: Orchestrating Multi-scale & Hybrid Architecture for Image Manipulation Localization](https://mlsanity.com/paper/2412.13753/) (2024)
- [Generate, Segment and Refine: Towards Generic Manipulation Segmentation](https://mlsanity.com/paper/1811.09729/) (2018)
- [Can We Get Rid of Handcrafted Feature Extractors? SparseViT: Nonsemantics-Centered, Parameter-Efficient Image Manipulation Localization through Spare-Coding Transformer](https://mlsanity.com/paper/2412.14598/) (2024)
- [TransCAM: Transformer Attention-based CAM Refinement for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.07239/) (2022)
- [Towards Generic Image Manipulation Detection with Weakly-Supervised Self-Consistency Learning](https://mlsanity.com/paper/2309.01246/) (2023)
- [Towards More Unified In-context Visual Understanding](https://mlsanity.com/paper/2312.02520/) (2023)
- [PROMPT-IML: Image Manipulation Localization with Pre-trained Foundation Models Through Prompt Tuning](https://mlsanity.com/paper/2401.00653/) (2024)

## Built on by (most cited first, 2 of 2)

- [Training-Free In-Context Forensic Chain for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2510.10111/) (2025)
- [SAPL: Semantic-Agnostic Prompt Learning in CLIP for Weakly Supervised Image Manipulation Localization](https://mlsanity.com/paper/2601.06222/) (2026)

## Abstract (arXiv)

Malicious image manipulation poses societal risks, increasing the importance of effective image manipulation detection methods. Recent approaches in image manipulation detection have largely been driven by fully supervised approaches, which require labor-intensive pixel-level annotations. Thus, it is essential to explore weakly supervised image manipulation localization methods that only require image-level binary labels for training. However, existing weakly supervised image manipulation methods overlook the importance of edge information for accurate localization, leading to suboptimal localization performance. To address this, we propose a Context-Aware Boundary Localization (CABL) module to aggregate boundary features and learn context-inconsistency for localizing manipulated areas. Furthermore, by leveraging Class Activation Mapping (CAM) and Segment Anything Model (SAM), we introduce the CAM-Guided SAM Refinement (CGSR) module to generate more accurate manipulation localization maps. By integrating two modules, we present a novel weakly supervised framework based on a dual-branch Transformer-CNN architecture. Our method achieves outstanding localization performance across multiple datasets.