# Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks

arXiv: https://arxiv.org/abs/2310.00076 · HTML: https://mlsanity.com/paper/2310.00076/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 34 of its 59 references that are in the map.
- First posted on arXiv 29 Sep 2023.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [StegaStamp: Invisible Hyperlinks in Physical Photographs](https://mlsanity.com/paper/1904.05343/) (2019)
- [Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust](https://mlsanity.com/paper/2305.20030/) (2023)
- [Robust Invisible Video Watermarking with Attention](https://mlsanity.com/paper/1909.01285/) (2019)
- [FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals](https://mlsanity.com/paper/1901.02212/) (2019)
- [A Recipe for Watermarking Diffusion Models](https://mlsanity.com/paper/2303.10137/) (2023)
- [Invisible Image Watermarks Are Provably Removable Using Generative AI](https://mlsanity.com/paper/2306.01953/) (2023)
- [Two-Stream Neural Networks for Tampered Face Detection](https://mlsanity.com/paper/1803.11276/) (2018)
- [MBRS : Enhancing Robustness of DNN-based Watermarking by Mini-Batch of Real and Simulated JPEG Compression](https://mlsanity.com/paper/2108.08211/) (2021)
- [Evading Watermark based Detection of AI-Generated Content](https://mlsanity.com/paper/2305.03807/) (2023)
- [SteganoGAN: High Capacity Image Steganography with GANs](https://mlsanity.com/paper/1901.03892/) (2019)
- [Image Denoising: The Deep Learning Revolution and Beyond -- A Survey Paper --](https://mlsanity.com/paper/2301.03362/) (2023)
- [Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection](https://mlsanity.com/paper/1703.04615/) (2017)
- [Sharp Multiple Instance Learning for DeepFake Video Detection](https://mlsanity.com/paper/2008.04585/) (2020)
- [DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Models](https://mlsanity.com/paper/2306.04642/) (2023)

## Built on by (most cited first, 30 of 81)

- [WAVES: Benchmarking the Robustness of Image Watermarks](https://mlsanity.com/paper/2401.08573/) (2024)
- [Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models](https://mlsanity.com/paper/2311.04378/) (2023)
- [An Undetectable Watermark for Generative Image Models](https://mlsanity.com/paper/2410.07369/) (2024)
- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [Black-Box Forgery Attacks on Semantic Watermarks for Diffusion Models](https://mlsanity.com/paper/2412.03283/) (2024)
- [SoK: Watermarking for AI-Generated Content](https://mlsanity.com/paper/2411.18479/) (2024)
- [Image Watermarks are Removable Using Controllable Regeneration from Clean Noise](https://mlsanity.com/paper/2410.05470/) (2024)
- [On the Challenges and Opportunities in Generative AI](https://mlsanity.com/paper/2403.00025/) (2024)
- [WMAdapter: Adding WaterMark Control to Latent Diffusion Models](https://mlsanity.com/paper/2406.08337/) (2024)
- [SynthID-Image: Image watermarking at internet scale](https://mlsanity.com/paper/2510.09263/) (2025)
- [A Transfer Attack to Image Watermarks](https://mlsanity.com/paper/2403.15365/) (2024)
- [UnMarker: A Universal Attack on Defensive Image Watermarking](https://mlsanity.com/paper/2405.08363/) (2024)
- [InvisMark: Invisible and Robust Watermarking for AI-generated Image Provenance](https://mlsanity.com/paper/2411.07795/) (2024)
- [IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generation](https://mlsanity.com/paper/2412.11638/) (2024)
- [Stable Signature is Unstable: Removing Image Watermark from Diffusion Models](https://mlsanity.com/paper/2405.07145/) (2024)
- [Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text](https://mlsanity.com/paper/2506.07001/) (2025)
- [Watermarking Autoregressive Image Generation](https://mlsanity.com/paper/2506.16349/) (2025)
- [A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authentication](https://mlsanity.com/paper/2407.10575/) (2024)
- [Forging and Removing Latent-Noise Diffusion Watermarks Using a Single Image](https://mlsanity.com/paper/2504.20111/) (2025)
- [Transferable Black-Box One-Shot Forging of Watermarks via Image Preference Models](https://mlsanity.com/paper/2510.20468/) (2025)
- [The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources](https://mlsanity.com/paper/2406.16746/) (2024)
- [AI Safety Should Prioritize the Future of Work](https://mlsanity.com/paper/2504.13959/) (2025)
- [OpenFake: An Open Dataset and Platform Toward Real-World Deepfake Detection](https://mlsanity.com/paper/2509.09495/) (2025)
- [JIGMARK: A Black-Box Approach for Enhancing Image Watermarks against Diffusion Model Edits](https://mlsanity.com/paper/2406.03720/) (2024)
- [FreqMark: Invisible Image Watermarking via Frequency Based Optimization in Latent Space](https://mlsanity.com/paper/2410.20824/) (2024)
- [AI-generated Image Detection: Passive or Watermark?](https://mlsanity.com/paper/2411.13553/) (2024)
- [A Baseline Method for Removing Invisible Image Watermarks using Deep Image Prior](https://mlsanity.com/paper/2502.13998/) (2025)
- [WMCopier: Forging Invisible Image Watermarks on Arbitrary Images](https://mlsanity.com/paper/2503.22330/) (2025)
- [VideoMarkBench: Benchmarking Robustness of Video Watermarking](https://mlsanity.com/paper/2505.21620/) (2025)
- [BitMark: Watermarking Bitwise Autoregressive Image Generative Models](https://mlsanity.com/paper/2506.21209/) (2025)

## Abstract (arXiv)

In light of recent advancements in generative AI models, it has become essential to distinguish genuine content from AI-generated one to prevent the malicious usage of fake materials as authentic ones and vice versa. Various techniques have been introduced for identifying AI-generated images, with watermarking emerging as a promising approach. In this paper, we analyze the robustness of various AI-image detectors including watermarking and classifier-based deepfake detectors. For watermarking methods that introduce subtle image perturbations (i.e., low perturbation budget methods), we reveal a fundamental trade-off between the evasion error rate (i.e., the fraction of watermarked images detected as non-watermarked ones) and the spoofing error rate (i.e., the fraction of non-watermarked images detected as watermarked ones) upon an application of diffusion purification attack. To validate our theoretical findings, we also provide empirical evidence demonstrating that diffusion purification effectively removes low perturbation budget watermarks by applying minimal changes to images. The diffusion purification attack is ineffective for high perturbation watermarking methods where notable changes are applied to images. In this case, we develop a model substitution adversarial attack that can successfully remove watermarks. Moreover, we show that watermarking methods are vulnerable to spoofing attacks where the attacker aims to have real images identified as watermarked ones, damaging the reputation of the developers. In particular, with black-box access to the watermarking method, a watermarked noise image can be generated and added to real images, causing them to be incorrectly classified as watermarked. Finally, we extend our theory to characterize a fundamental trade-off between the robustness and reliability of classifier-based deep fake detectors and demonstrate it through experiments.