# OpenSDI: Spotting Diffusion-Generated Images in the Open World

arXiv: https://arxiv.org/abs/2503.19653 · HTML: https://mlsanity.com/paper/2503.19653/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2025.

## Built on by (most cited first, 23 of 23)

- [UniShield: An Adaptive Multi-Agent Framework for Unified Forgery Image Detection and Localization](https://mlsanity.com/paper/2510.03161/) (2025)
- [Venus-DeFakerOne: Unified Fake Image Detection & Localization](https://mlsanity.com/paper/2605.14091/) (2026)
- [Towards Reliable Identification of Diffusion-based Image Manipulations](https://mlsanity.com/paper/2506.05466/) (2025)
- [Webly-Supervised Image Manipulation Localization via Category-Aware Auto-Annotation](https://mlsanity.com/paper/2508.20987/) (2025)
- [Revisiting Image Manipulation Localization under Realistic Manipulation Scenarios](https://mlsanity.com/paper/2509.20006/) (2025)
- [Can We Build a Monolithic Model for Fake Image Detection? SICA: Semantic-Induced Constrained Adaptation for Unified-Yet-Discriminative Artifact Feature Space Reconstruction](https://mlsanity.com/paper/2602.06676/) (2026)
- [DailyBench: A Unified Benchmark for AI-Generated and Manipulated Images from Modern Generative Models](https://mlsanity.com/paper/2607.24016/) (2026)
- [GAMMA: Generalizable Alignment via Multi-task and Manipulation-Augmented Training for AI-Generated Image Detection](https://mlsanity.com/paper/2509.10250/) (2025)
- [NeXT-IMDL: Build Benchmark for NeXT-Generation Image Manipulation Detection & Localization](https://mlsanity.com/paper/2512.23374/) (2025)
- [Leave No Stone Unturned: Uncovering Holistic Audio-Visual Intrinsic Coherence for Deepfake Detection](https://mlsanity.com/paper/2603.23960/) (2026)
- [TAP into the Patch Tokens: Leveraging Vision Foundation Model Features for AI-Generated Image Detection](https://mlsanity.com/paper/2604.26772/) (2026)
- [Impostor: An Agent-Curated Benchmark for Realistic AIGC Manipulation Localization](https://mlsanity.com/paper/2606.04545/) (2026)
- [LaP-Forensics: Latent-Pixel Consistency Guided Multimodal Reasoning for Deepfake Detection](https://mlsanity.com/paper/2607.25962/) (2026)
- [PATE-Forensics: Perception-as-Tool for Explainable Deepfake Forensics with General-Purpose MLLMs](https://mlsanity.com/paper/2608.18573/) (2026)
- [VTONGuard: Automatic Detection and Authentication of AI-Generated Virtual Try-On Content](https://mlsanity.com/paper/2601.13951/) (2026)
- [Detecting Diffusion-generated Images via Dynamic Assembly Forests](https://mlsanity.com/paper/2604.09106/) (2026)
- [DiffusionPrint: Learning Generative Fingerprints for Diffusion-Based Inpainting Localization](https://mlsanity.com/paper/2604.12443/) (2026)
- [OmniVL-Guard Pro: A Tool-Augmented Agent for Omnibus Vision-Language Forensics](https://mlsanity.com/paper/2605.16962/) (2026)
- [SIGMA: Semantic-Difference Instruction-Grounding Mask Annotator for Text-Driven Image Manipulation Localization](https://mlsanity.com/paper/2605.27924/) (2026)
- [TruEye: Fine-Grained Detection of AI-Generated Human Subjects in Images](https://mlsanity.com/paper/2606.27505/) (2026)
- [FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization](https://mlsanity.com/paper/2608.28302/) (2026)
- [Less Supervision, Better Generalization: Weakly Supervised Fake Region Localization in Diffusion-Edited Images](https://mlsanity.com/paper/2609.36882/) (2026)
- [GIFTBench: Diagnosing Generalization in Image Forgery Localization and Informing Model Design](https://mlsanity.com/paper/2610.01778/) (2026)

## Abstract (arXiv)

This paper identifies OpenSDI, a challenge for spotting diffusion-generated images in open-world settings. In response to this challenge, we define a new benchmark, the OpenSDI dataset (OpenSDID), which stands out from existing datasets due to its diverse use of large vision-language models that simulate open-world diffusion-based manipulations. Another outstanding feature of OpenSDID is its inclusion of both detection and localization tasks for images manipulated globally and locally by diffusion models. To address the OpenSDI challenge, we propose a Synergizing Pretrained Models (SPM) scheme to build up a mixture of foundation models. This approach exploits a collaboration mechanism with multiple pretrained foundation models to enhance generalization in the OpenSDI context, moving beyond traditional training by synergizing multiple pretrained models through prompting and attending strategies. Building on this scheme, we introduce MaskCLIP, an SPM-based model that aligns Contrastive Language-Image Pre-Training (CLIP) with Masked Autoencoder (MAE). Extensive evaluations on OpenSDID show that MaskCLIP significantly outperforms current state-of-the-art methods for the OpenSDI challenge, achieving remarkable relative improvements of 14.23% in IoU (14.11% in F1) and 2.05% in accuracy (2.38% in F1) compared to the second-best model in localization and detection tasks, respectively. Our dataset and code are available at https://github.com/iamwangyabin/OpenSDI.