# ForgeryTTT: Zero-Shot Image Manipulation Localization with Test-Time Training

arXiv: https://arxiv.org/abs/2410.04032 · HTML: https://mlsanity.com/paper/2410.04032/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 5 Oct 2024.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [FateZero: Fusing Attentions for Zero-shot Text-based Video Editing](https://mlsanity.com/paper/2303.09535/) (2023)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Test-Time Training with Masked Autoencoders](https://mlsanity.com/paper/2209.07522/) (2022)
- [SmartEdit: Exploring Complex Instruction-based Image Editing with Multimodal Large Language Models](https://mlsanity.com/paper/2312.06739/) (2023)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2103.10596/) (2021)
- [The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization](https://mlsanity.com/paper/2112.00463/) (2021)
- [TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization](https://mlsanity.com/paper/2212.10957/) (2022)
- [Noiseprint: a CNN-based camera model fingerprint](https://mlsanity.com/paper/1808.08396/) (2018)
- [Explicit Visual Prompting for Low-Level Structure Segmentations](https://mlsanity.com/paper/2303.10883/) (2023)
- [Image Manipulation Detection by Multi-View Multi-Scale Supervision](https://mlsanity.com/paper/2104.06832/) (2021)
- [SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization](https://mlsanity.com/paper/2009.00726/) (2020)
- [Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2108.12947/) (2021)
- [ObjectFormer for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2203.14681/) (2022)
- [Improving the Harmony of the Composite Image by Spatial-Separated Attention Module](https://mlsanity.com/paper/1907.06406/) (2019)
- [Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries](https://mlsanity.com/paper/1903.02495/) (2019)
- [CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying](https://mlsanity.com/paper/2303.08524/) (2023)
- [Test-Time Personalization with a Transformer for Human Pose Estimation](https://mlsanity.com/paper/2107.02133/) (2021)
- [Depth-aware Test-Time Training for Zero-shot Video Object Segmentation](https://mlsanity.com/paper/2403.04258/) (2024)
- [Explicit Visual Prompting for Universal Foreground Segmentations](https://mlsanity.com/paper/2305.18476/) (2023)
- [Two-Stage Copy-Move Forgery Detection with Self Deep Matching and Proposal SuperGlue](https://mlsanity.com/paper/2012.08697/) (2020)

## Built on by (most cited first, 8 of 8)

- [Underwater Image Restoration via Polymorphic Large Kernel CNNs](https://mlsanity.com/paper/2412.18459/) (2024)
- [High-Fidelity Document Stain Removal via A Large-Scale Real-World Dataset and A Memory-Augmented Transformer](https://mlsanity.com/paper/2410.22922/) (2024)
- [AnimeDL-2M: Million-Scale AI-Generated Anime Image Detection and Localization in Diffusion Era](https://mlsanity.com/paper/2504.11015/) (2025)
- [MAC-Lookup: Multi-Axis Conditional Lookup Model for Underwater Image Enhancement](https://mlsanity.com/paper/2507.02270/) (2025)
- [SFormer: SNR-guided Transformer for Underwater Image Enhancement from the Frequency Domain](https://mlsanity.com/paper/2508.18664/) (2025)
- [A Comprehensive Analysis of Churn Prediction in Telecommunications Using Machine Learning](https://mlsanity.com/paper/2509.22654/) (2025)
- [Forestpest-YOLO: A High-Performance Detection Framework for Small Forestry Pests](https://mlsanity.com/paper/2510.00547/) (2025)
- [VEDAL: Variational Error-Driven Asynchronous Learning for 3D Gaussian Splatting Pruning](https://mlsanity.com/paper/2606.02346/) (2026)

## Abstract (arXiv)

Social media is increasingly plagued by realistic fake images, making it hard to trust content. Previous algorithms to detect these fakes often fail in new, real-world scenarios because they are trained on specific datasets. To address the problem, we introduce ForgeryTTT, the first method leveraging test-time training (TTT) to identify manipulated regions in images. The proposed approach fine-tunes the model for each individual test sample, improving its performance. ForgeryTTT first employs vision transformers as a shared image encoder to learn both classification and localization tasks simultaneously during the training-time training using a large synthetic dataset. Precisely, the localization head predicts a mask to highlight manipulated areas. Given such a mask, the input tokens can be divided into manipulated and genuine groups, which are then fed into the classification head to distinguish between manipulated and genuine parts. During test-time training, the predicted mask from the localization head is used for the classification head to update the image encoder for better adaptation. Additionally, using the classical dropout strategy in each token group significantly improves performance and efficiency. We test ForgeryTTT on five standard benchmarks. Despite its simplicity, ForgeryTTT achieves a 20.1% improvement in localization accuracy compared to other zero-shot methods and a 4.3% improvement over non-zero-shot techniques. Our code and data will be released upon publication.