# Loupe: A Generalizable and Adaptive Framework for Image Forgery Detection

arXiv: https://arxiv.org/abs/2506.16819 · HTML: https://mlsanity.com/paper/2506.16819/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 23 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Builds on (most cited first, 20 of 20)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://mlsanity.com/paper/2404.06395/) (2024)
- [Perception Encoder: The best visual embeddings are not at the output of the network](https://mlsanity.com/paper/2504.13181/) (2025)
- [Tversky loss function for image segmentation using 3D fully convolutional deep networks](https://mlsanity.com/paper/1706.05721/) (2017)
- [DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection](https://mlsanity.com/paper/2307.01426/) (2023)
- [Hierarchical Fine-Grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2303.17111/) (2023)
- [SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model](https://mlsanity.com/paper/2412.04292/) (2024)
- [Learning JPEG Compression Artifacts for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2108.12947/) (2021)
- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [Preserving Fairness Generalization in Deepfake Detection](https://mlsanity.com/paper/2402.17229/) (2024)
- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [LEGION: Learning to Ground and Explain for Synthetic Image Detection](https://mlsanity.com/paper/2503.15264/) (2025)
- [PolyLoss: A Polynomial Expansion Perspective of Classification Loss Functions](https://mlsanity.com/paper/2204.12511/) (2022)
- [Your ViT is Secretly an Image Segmentation Model](https://mlsanity.com/paper/2503.19108/) (2025)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [A Survey of Multimodal-Guided Image Editing with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2406.14555/) (2024)

## Abstract (arXiv)

The proliferation of generative models has raised serious concerns about visual content forgery. Existing deepfake detection methods primarily target either image-level classification or pixel-wise localization. While some achieve high accuracy, they often suffer from limited generalization across manipulation types or rely on complex architectures. In this paper, we propose Loupe, a lightweight yet effective framework for joint deepfake detection and localization. Loupe integrates a patch-aware classifier and a segmentation module with conditional queries, allowing simultaneous global authenticity classification and fine-grained mask prediction. To enhance robustness against distribution shifts of test set, Loupe introduces a pseudo-label-guided test-time adaptation mechanism by leveraging patch-level predictions to supervise the segmentation head. Extensive experiments on the DDL dataset demonstrate that Loupe achieves state-of-the-art performance, securing the first place in the IJCAI 2025 Deepfake Detection and Localization Challenge with an overall score of 0.846. Our results validate the effectiveness of the proposed patch-level fusion and conditional query design in improving both classification accuracy and spatial localization under diverse forgery patterns. The code is available at https://github.com/Kamichanw/Loupe.