# SIDBench: A Python Framework for Reliably Assessing Synthetic Image Detection Methods

arXiv: https://arxiv.org/abs/2404.18552 · HTML: https://mlsanity.com/paper/2404.18552/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 29 Apr 2024.

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020)
- [Towards Universal Fake Image Detectors that Generalize Across Generative Models](https://mlsanity.com/paper/2302.10174/) (2023)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [DIRE for Diffusion-Generated Image Detection](https://mlsanity.com/paper/2303.09295/) (2023)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [Global Texture Enhancement for Fake Face Detection in the Wild](https://mlsanity.com/paper/2002.00133/) (2020)
- [DE-FAKE: Detection and Attribution of Fake Images Generated by Text-to-Image Generation Models](https://mlsanity.com/paper/2210.06998/) (2022)
- [PatchCraft: Exploring Texture Patch for Efficient AI-generated Image Detection](https://mlsanity.com/paper/2311.12397/) (2023)
- [AUC Maximization in the Era of Big Data and AI: A Survey](https://mlsanity.com/paper/2203.15046/) (2022)
- [Leveraging Representations from Intermediate Encoder-blocks for Synthetic Image Detection](https://mlsanity.com/paper/2402.19091/) (2024)

## Built on by (most cited first, 14 of 14)

- [Any-Resolution AI-Generated Image Detection by Spectral Learning](https://mlsanity.com/paper/2411.19417/) (2024)
- [TGIF: Text-Guided Inpainting Forgery Dataset](https://mlsanity.com/paper/2407.11566/) (2024)
- [AI-GenBench: A New Ongoing Benchmark for AI-Generated Image Detection](https://mlsanity.com/paper/2504.20865/) (2025)
- [Navigating the Challenges of AI-Generated Image Detection in the Wild: What Truly Matters?](https://mlsanity.com/paper/2507.10236/) (2025)
- [TextureCrop: Enhancing Synthetic Image Detection through Texture-based Cropping](https://mlsanity.com/paper/2407.15500/) (2024)
- [Evolution of Detection Performance throughout the Online Lifespan of Synthetic Images](https://mlsanity.com/paper/2408.11541/) (2024)
- [Present and Future Generalization of Synthetic Image Detectors](https://mlsanity.com/paper/2409.14128/) (2024)
- [UniAIDet: A Unified and Universal Benchmark for AI-Generated Image Content Detection and Localization](https://mlsanity.com/paper/2510.23023/) (2025)
- [Generalized Design Choices for Deepfake Detectors](https://mlsanity.com/paper/2511.21507/) (2025)
- [ForensicConcept: Transferable Forensic Concepts for AIGI Detection](https://mlsanity.com/paper/2606.07034/) (2026)
- [From Masks to Pixels and Meaning: A New Taxonomy, Benchmark, and Metrics for VLM Image Tampering](https://mlsanity.com/paper/2603.20193/) (2026)
- [Composite Data Augmentations for Synthetic Image Detection Against Real-World Perturbations](https://mlsanity.com/paper/2506.11490/) (2025)
- [Multi-Feature Fusion Approach for Generative AI Images Detection](https://mlsanity.com/paper/2603.29788/) (2026)
- [Automated In-the-Wild Data Collection for Continual AI Generated Image Detection](https://mlsanity.com/paper/2605.02567/) (2026)

## Abstract (arXiv)

The generative AI technology offers an increasing variety of tools for generating entirely synthetic images that are increasingly indistinguishable from real ones. Unlike methods that alter portions of an image, the creation of completely synthetic images presents a unique challenge and several Synthetic Image Detection (SID) methods have recently appeared to tackle it. Yet, there is often a large gap between experimental results on benchmark datasets and the performance of methods in the wild. To better address the evaluation needs of SID and help close this gap, this paper introduces a benchmarking framework that integrates several state-of-the-art SID models. Our selection of integrated models was based on the utilization of varied input features, and different network architectures, aiming to encompass a broad spectrum of techniques. The framework leverages recent datasets with a diverse set of generative models, high level of photo-realism and resolution, reflecting the rapid improvements in image synthesis technology. Additionally, the framework enables the study of how image transformations, common in assets shared online, such as JPEG compression, affect detection performance. SIDBench is available on https://github.com/mever-team/sidbench and is designed in a modular manner to enable easy inclusion of new datasets and SID models.