# MMFusion: Combining Image Forensic Filters for Visual Manipulation Detection and Localization

arXiv: https://arxiv.org/abs/2312.01790 · HTML: https://mlsanity.com/paper/2312.01790/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 57 references that are in the map.
- First posted on arXiv 4 Dec 2023.

## Builds on (most cited first, 27 of 27)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Early Convolutions Help Transformers See Better](https://mlsanity.com/paper/2106.14881/) (2021)
- [What Makes Training Multi-Modal Classification Networks Hard?](https://mlsanity.com/paper/1905.12681/) (2019)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Model](https://mlsanity.com/paper/2212.05034/) (2022)
- [Generalizing Face Forgery Detection with High-frequency Features](https://mlsanity.com/paper/2103.12376/) (2021)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [UCF: Uncovering Common Features for Generalizable Deepfake Detection](https://mlsanity.com/paper/2304.13949/) (2023)
- [Learning Rich Features for Image Manipulation Detection](https://mlsanity.com/paper/1805.04953/) (2018)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization](https://mlsanity.com/paper/2212.10957/) (2022)
- [Delivering Arbitrary-Modal Semantic Segmentation](https://mlsanity.com/paper/2303.01480/) (2023)
- [PMR: Prototypical Modal Rebalance for Multimodal Learning](https://mlsanity.com/paper/2211.07089/) (2022)
- [Noiseprint: a CNN-based camera model fingerprint](https://mlsanity.com/paper/1808.08396/) (2018)
- [Image Manipulation Detection by Multi-View Multi-Scale Supervision](https://mlsanity.com/paper/2104.06832/) (2021)
- [SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization](https://mlsanity.com/paper/2009.00726/) (2020)
- [Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2108.12947/) (2021)
- [ObjectFormer for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2203.14681/) (2022)
- [Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)](https://mlsanity.com/paper/1709.02016/) (2017)
- [Constrained R-CNN: A general image manipulation detection model](https://mlsanity.com/paper/1911.08217/) (2019)
- [VideoFACT: Detecting Video Forgeries Using Attention, Scene Context, and Forensic Traces](https://mlsanity.com/paper/2211.15775/) (2022)

## Built on by (most cited first, 5 of 5)

- [REALEDIT: Reddit Edits As a Large-scale Empirical Dataset for Image Transformations](https://mlsanity.com/paper/2502.03629/) (2025)
- [Deep Image Composition Meets Image Forgery](https://mlsanity.com/paper/2404.02897/) (2024)
- [Semantic Manipulation Localization](https://mlsanity.com/paper/2604.10132/) (2026)
- [Deep Image Restoration For Image Anti-Forensics](https://mlsanity.com/paper/2405.02751/) (2024)
- [DiffFace-Edit: A Diffusion-Based Facial Dataset for Forgery-Semantic Driven Deepfake Detection Analysis](https://mlsanity.com/paper/2601.13551/) (2026)

## Abstract (arXiv)

Recent image manipulation localization and detection techniques typically leverage forensic artifacts and traces that are produced by a noise-sensitive filter, such as SRM or Bayar convolution. In this paper, we showcase that different filters commonly used in such approaches excel at unveiling different types of manipulations and provide complementary forensic traces. Thus, we explore ways of combining the outputs of such filters to leverage the complementary nature of the produced artifacts for performing image manipulation localization and detection (IMLD). We assess two distinct combination methods: one that produces independent features from each forensic filter and then fuses them (this is referred to as late fusion) and one that performs early mixing of different modal outputs and produces combined features (this is referred to as early fusion). We use the latter as a feature encoding mechanism, accompanied by a new decoding mechanism that encompasses feature re-weighting, for formulating the proposed MMFusion architecture. We demonstrate that MMFusion achieves competitive performance for both image manipulation localization and detection, outperforming state-of-the-art models across several image and video datasets. We also investigate further the contribution of each forensic filter within MMFusion for addressing different types of manipulations, building on recent AI explainability measures.