# Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries

arXiv: https://arxiv.org/abs/1903.02495 · HTML: https://mlsanity.com/paper/1903.02495/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2019.

## Built on by (most cited first, 30 of 59)

- [Exploring Temporal Coherence for More General Video Face Forgery Detection](https://mlsanity.com/paper/2108.06693/) (2021)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [MVSS-Net: Multi-View Multi-Scale Supervised Networks for Image Manipulation Detection](https://mlsanity.com/paper/2112.08935/) (2021)
- [Protecting Celebrities from DeepFake with Identity Consistency Transformer](https://mlsanity.com/paper/2203.01318/) (2022)
- [Face Forensics in the Wild](https://mlsanity.com/paper/2103.16076/) (2021)
- [ObjectFormer for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2203.14681/) (2022)
- [AltFreezing for More General Video Face Forgery Detection](https://mlsanity.com/paper/2307.08317/) (2023)
- [FrePGAN: Robust Deepfake Detection Using Frequency-level Perturbations](https://mlsanity.com/paper/2202.03347/) (2022)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [BiHPF: Bilateral High-Pass Filters for Robust Deepfake Detection](https://mlsanity.com/paper/2109.00911/) (2021)
- [FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learning](https://mlsanity.com/paper/2105.13617/) (2021)
- [CoReD: Generalizing Fake Media Detection with Continual Representation using Distillation](https://mlsanity.com/paper/2107.02408/) (2021)
- [Pixel-Inconsistency Modeling for Image Manipulation Localization](https://mlsanity.com/paper/2310.00234/) (2023)
- [Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors](https://mlsanity.com/paper/2109.02993/) (2021)
- [TransForensics: Image Forgery Localization with Dense Self-Attention](https://mlsanity.com/paper/2108.03871/) (2021)
- [Self-Adversarial Training incorporating Forgery Attention for Image Forgery Localization](https://mlsanity.com/paper/2107.02434/) (2021)
- [Weakly-supervised deepfake localization in diffusion-generated images](https://mlsanity.com/paper/2311.04584/) (2023)
- [Detection and Localization of Facial Expression Manipulations](https://mlsanity.com/paper/2103.08134/) (2021)
- [Exploring Multi-view Pixel Contrast for General and Robust Image Forgery Localization](https://mlsanity.com/paper/2406.13565/) (2024)
- [ForgeryGPT: A Multimodal LLM for Interpretable Image Forgery Detection and Localization](https://mlsanity.com/paper/2410.10238/) (2024)
- [CFL-Net: Image Forgery Localization Using Contrastive Learning](https://mlsanity.com/paper/2210.02182/) (2022)
- [UPOCR: Towards Unified Pixel-Level OCR Interface](https://mlsanity.com/paper/2312.02694/) (2023)
- [GCA-Net : Utilizing Gated Context Attention for Improving Image Forgery Localization and Detection](https://mlsanity.com/paper/2112.04298/) (2021)
- [Towards Effective Image Manipulation Detection with Proposal Contrastive Learning](https://mlsanity.com/paper/2210.08529/) (2022)
- [ForgeryTTT: Zero-Shot Image Manipulation Localization with Test-Time Training](https://mlsanity.com/paper/2410.04032/) (2024)
- [Fusion Transformer with Object Mask Guidance for Image Forgery Analysis](https://mlsanity.com/paper/2403.12229/) (2024)
- [Inharmonious Region Localization](https://mlsanity.com/paper/2104.09453/) (2021)
- [TrainFors: A Large Benchmark Training Dataset for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2308.05264/) (2023)
- [TBNet:Two-Stream Boundary-aware Network for Generic Image Manipulation Localization](https://mlsanity.com/paper/2108.04508/) (2021)
- [Effective Image Tampering Localization with Multi-Scale ConvNeXt Feature Fusion](https://mlsanity.com/paper/2208.13739/) (2022)

## Abstract (arXiv)

With advanced image journaling tools, one can easily alter the semantic meaning of an image by exploiting certain manipulation techniques such as copy-clone, object splicing, and removal, which mislead the viewers. In contrast, the identification of these manipulations becomes a very challenging task as manipulated regions are not visually apparent. This paper proposes a high-confidence manipulation localization architecture which utilizes resampling features, Long-Short Term Memory (LSTM) cells, and encoder-decoder network to segment out manipulated regions from non-manipulated ones. Resampling features are used to capture artifacts like JPEG quality loss, upsampling, downsampling, rotation, and shearing. The proposed network exploits larger receptive fields (spatial maps) and frequency domain correlation to analyze the discriminative characteristics between manipulated and non-manipulated regions by incorporating encoder and LSTM network. Finally, decoder network learns the mapping from low-resolution feature maps to pixel-wise predictions for image tamper localization. With predicted mask provided by final layer (softmax) of the proposed architecture, end-to-end training is performed to learn the network parameters through back-propagation using ground-truth masks. Furthermore, a large image splicing dataset is introduced to guide the training process. The proposed method is capable of localizing image manipulations at pixel level with high precision, which is demonstrated through rigorous experimentation on three diverse datasets.