# ObjectFormer for Image Manipulation Detection and Localization

arXiv: https://arxiv.org/abs/2203.14681 · HTML: https://mlsanity.com/paper/2203.14681/

## Key facts

- Cited by 78 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 48 references that are in the map.
- First posted on arXiv 28 Mar 2022.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Early Convolutions Help Transformers See Better](https://mlsanity.com/paper/2106.14881/) (2021)
- [Rethinking Spatial Dimensions of Vision Transformers](https://mlsanity.com/paper/2103.16302/) (2021)
- [AdaViT: Adaptive Vision Transformers for Efficient Image Recognition](https://mlsanity.com/paper/2111.15668/) (2021)
- [Swapping Autoencoder for Deep Image Manipulation](https://mlsanity.com/paper/2007.00653/) (2020)
- [BEVT: BERT Pretraining of Video Transformers](https://mlsanity.com/paper/2112.01529/) (2021)
- [ManiGAN: Text-Guided Image Manipulation](https://mlsanity.com/paper/1912.06203/) (2019)
- [Learning Rich Features for Image Manipulation Detection](https://mlsanity.com/paper/1805.04953/) (2018)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection](https://mlsanity.com/paper/2104.09770/) (2021)
- [Local Relation Learning for Face Forgery Detection](https://mlsanity.com/paper/2105.02577/) (2021)
- [SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization](https://mlsanity.com/paper/2009.00726/) (2020)
- [Semantic Image Manipulation Using Scene Graphs](https://mlsanity.com/paper/2004.03677/) (2020)
- [Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries](https://mlsanity.com/paper/1903.02495/) (2019)

## Built on by (most cited first, 30 of 78)

- [Hierarchical Fine-Grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2303.17111/) (2023)
- [TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization](https://mlsanity.com/paper/2212.10957/) (2022)
- [OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation](https://mlsanity.com/paper/2406.09399/) (2024)
- [Explicit Visual Prompting for Low-Level Structure Segmentations](https://mlsanity.com/paper/2303.10883/) (2023)
- [Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning](https://mlsanity.com/paper/2207.04978/) (2022)
- [AutoSplice: A Text-prompt Manipulated Image Dataset for Media Forensics](https://mlsanity.com/paper/2304.06870/) (2023)
- [IML-ViT: Benchmarking Image Manipulation Localization by Vision Transformer](https://mlsanity.com/paper/2307.14863/) (2023)
- [IMDL-BenCo: A Comprehensive Benchmark and Codebase for Image Manipulation Detection & Localization](https://mlsanity.com/paper/2406.10580/) (2024)
- [Look Before You Match: Instance Understanding Matters in Video Object Segmentation](https://mlsanity.com/paper/2212.06826/) (2022)
- [Pixel-Inconsistency Modeling for Image Manipulation Localization](https://mlsanity.com/paper/2310.00234/) (2023)
- [Pre-training-free Image Manipulation Localization through Non-Mutually Exclusive Contrastive Learning](https://mlsanity.com/paper/2309.14900/) (2023)
- [Toward Real Text Manipulation Detection: New Dataset and New Solution](https://mlsanity.com/paper/2312.06934/) (2023)
- [GIM: A Million-scale Benchmark for Generative Image Manipulation Detection and Localization](https://mlsanity.com/paper/2406.16531/) (2024)
- [Explicit Visual Prompting for Universal Foreground Segmentations](https://mlsanity.com/paper/2305.18476/) (2023)
- [Language-guided Hierarchical Fine-grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2410.23556/) (2024)
- [Exploring Multi-view Pixel Contrast for General and Robust Image Forgery Localization](https://mlsanity.com/paper/2406.13565/) (2024)
- [ForgeryGPT: A Multimodal LLM for Interpretable Image Forgery Detection and Localization](https://mlsanity.com/paper/2410.10238/) (2024)
- [Multi-spectral Class Center Network for Face Manipulation Detection and Localization](https://mlsanity.com/paper/2305.10794/) (2023)
- [ForgerySleuth: Empowering Multimodal Large Language Models for Image Manipulation Detection](https://mlsanity.com/paper/2411.19466/) (2024)
- [TBFormer: Two-Branch Transformer for Image Forgery Localization](https://mlsanity.com/paper/2302.13004/) (2023)
- [ForgeryTTT: Zero-Shot Image Manipulation Localization with Test-Time Training](https://mlsanity.com/paper/2410.04032/) (2024)
- [A Neurosymbolic Framework for Interpretable Cognitive Attack Detection in Augmented Reality](https://mlsanity.com/paper/2508.09185/) (2025)
- [Discrepancy-Guided Reconstruction Learning for Image Forgery Detection](https://mlsanity.com/paper/2304.13349/) (2023)
- [Mixture-of-Noises Enhanced Forgery-Aware Predictor for Multi-Face Manipulation Detection and Localization](https://mlsanity.com/paper/2408.02306/) (2024)
- [COMPrompter: reconceptualized segment anything model with multiprompt network for camouflaged object detection](https://mlsanity.com/paper/2411.18858/) (2024)
- [ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack](https://mlsanity.com/paper/2508.07402/) (2025)
- [ReLoc: A Restoration-Assisted Framework for Robust Image Tampering Localization](https://mlsanity.com/paper/2211.03930/) (2022)
- [SimpSON: Simplifying Photo Cleanup with Single-Click Distracting Object Segmentation Network](https://mlsanity.com/paper/2305.17624/) (2023)
- [TrainFors: A Large Benchmark Training Dataset for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2308.05264/) (2023)
- [MMFusion: Combining Image Forensic Filters for Visual Manipulation Detection and Localization](https://mlsanity.com/paper/2312.01790/) (2023)

## Abstract (arXiv)

Recent advances in image editing techniques have posed serious challenges to the trustworthiness of multimedia data, which drives the research of image tampering detection. In this paper, we propose ObjectFormer to detect and localize image manipulations. To capture subtle manipulation traces that are no longer visible in the RGB domain, we extract high-frequency features of the images and combine them with RGB features as multimodal patch embeddings. Additionally, we use a set of learnable object prototypes as mid-level representations to model the object-level consistencies among different regions, which are further used to refine patch embeddings to capture the patch-level consistencies. We conduct extensive experiments on various datasets and the results verify the effectiveness of the proposed method, outperforming state-of-the-art tampering detection and localization methods.