# WAVES: Benchmarking the Robustness of Image Watermarks

arXiv: https://arxiv.org/abs/2401.08573 · HTML: https://mlsanity.com/paper/2401.08573/

## Key facts

- Cited by 97 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 57 references that are in the map.
- First posted on arXiv 16 Jan 2024.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [HiDDeN: Hiding Data With Deep Networks](https://mlsanity.com/paper/1807.09937/) (2018)
- [DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models](https://mlsanity.com/paper/2210.14896/) (2022)
- [The Stable Signature: Rooting Watermarks in Latent Diffusion Models](https://mlsanity.com/paper/2303.15435/) (2023)
- [StegaStamp: Invisible Hyperlinks in Physical Photographs](https://mlsanity.com/paper/1904.05343/) (2019)
- [Adversarial Attacks and Defences: A Survey](https://mlsanity.com/paper/1810.00069/) (2018)
- [The Role of ImageNet Classes in Fr'echet Inception Distance](https://mlsanity.com/paper/2203.06026/) (2022)
- [Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data](https://mlsanity.com/paper/2007.08457/) (2020)
- [Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust](https://mlsanity.com/paper/2305.20030/) (2023)
- [How Robust is Google's Bard to Adversarial Image Attacks?](https://mlsanity.com/paper/2309.11751/) (2023)
- [Robust Invisible Video Watermarking with Attention](https://mlsanity.com/paper/1909.01285/) (2019)
- [Entangled Watermarks as a Defense against Model Extraction](https://mlsanity.com/paper/2002.12200/) (2020)
- [A Recipe for Watermarking Diffusion Models](https://mlsanity.com/paper/2303.10137/) (2023)
- [Invisible Image Watermarks Are Provably Removable Using Generative AI](https://mlsanity.com/paper/2306.01953/) (2023)
- [MBRS : Enhancing Robustness of DNN-based Watermarking by Mini-Batch of Real and Simulated JPEG Compression](https://mlsanity.com/paper/2108.08211/) (2021)
- [Watermarking Images in Self-Supervised Latent Spaces](https://mlsanity.com/paper/2112.09581/) (2021)

## Built on by (most cited first, 30 of 97)

- [Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances](https://mlsanity.com/paper/2410.18775/) (2024)
- [RingID: Rethinking Tree-Ring Watermarking for Enhanced Multi-Key Identification](https://mlsanity.com/paper/2404.14055/) (2024)
- [An Undetectable Watermark for Generative Image Models](https://mlsanity.com/paper/2410.07369/) (2024)
- [Black-Box Forgery Attacks on Semantic Watermarks for Diffusion Models](https://mlsanity.com/paper/2412.03283/) (2024)
- [SoK: Watermarking for AI-Generated Content](https://mlsanity.com/paper/2411.18479/) (2024)
- [Image Watermarks are Removable Using Controllable Regeneration from Clean Noise](https://mlsanity.com/paper/2410.05470/) (2024)
- [WMAdapter: Adding WaterMark Control to Latent Diffusion Models](https://mlsanity.com/paper/2406.08337/) (2024)
- [SynthID-Image: Image watermarking at internet scale](https://mlsanity.com/paper/2510.09263/) (2025)
- [A Transfer Attack to Image Watermarks](https://mlsanity.com/paper/2403.15365/) (2024)
- [UnMarker: A Universal Attack on Defensive Image Watermarking](https://mlsanity.com/paper/2405.08363/) (2024)
- [Hidden in the Noise: Two-Stage Robust Watermarking for Images](https://mlsanity.com/paper/2412.04653/) (2024)
- [Stable Signature is Unstable: Removing Image Watermark from Diffusion Models](https://mlsanity.com/paper/2405.07145/) (2024)
- [Watermarking Autoregressive Image Generation](https://mlsanity.com/paper/2506.16349/) (2025)
- [A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authentication](https://mlsanity.com/paper/2407.10575/) (2024)
- [Gaussian Shading++: Rethinking the Realistic Deployment Challenge of Performance-Lossless Image Watermark for Diffusion Models](https://mlsanity.com/paper/2504.15026/) (2025)
- [Transferable Black-Box One-Shot Forging of Watermarks via Image Preference Models](https://mlsanity.com/paper/2510.20468/) (2025)
- [Steganalysis on Digital Watermarking: Is Your Defense Truly Impervious?](https://mlsanity.com/paper/2406.09026/) (2024)
- [Watermarking Without Standards Is Not AI Governance](https://mlsanity.com/paper/2505.23814/) (2025)
- [First-Place Solution to NeurIPS 2024 Invisible Watermark Removal Challenge](https://mlsanity.com/paper/2508.21072/) (2025)
- [FreqMark: Invisible Image Watermarking via Frequency Based Optimization in Latent Space](https://mlsanity.com/paper/2410.20824/) (2024)
- [AI-generated Image Detection: Passive or Watermark?](https://mlsanity.com/paper/2411.13553/) (2024)
- [A Baseline Method for Removing Invisible Image Watermarks using Deep Image Prior](https://mlsanity.com/paper/2502.13998/) (2025)
- [VideoMarkBench: Benchmarking Robustness of Video Watermarking](https://mlsanity.com/paper/2505.21620/) (2025)
- [BitMark: Watermarking Bitwise Autoregressive Image Generative Models](https://mlsanity.com/paper/2506.21209/) (2025)
- [Secure and Robust Watermarking for AI-generated Images: A Comprehensive Survey](https://mlsanity.com/paper/2510.02384/) (2025)
- [Certifiably Robust Image Watermark](https://mlsanity.com/paper/2407.04086/) (2024)
- [Securing the Future of GenAI: Policy and Technology](https://mlsanity.com/paper/2407.12999/) (2024)
- [Secure and Efficient Watermarking for Latent Diffusion Models in Model Distribution Scenarios](https://mlsanity.com/paper/2502.13345/) (2025)
- [Visual Watermarking in the Era of Diffusion Models: Advances and Challenges](https://mlsanity.com/paper/2505.08197/) (2025)
- [A Crack in the Bark: Leveraging Public Knowledge to Remove Tree-Ring Watermarks](https://mlsanity.com/paper/2506.10502/) (2025)

## Abstract (arXiv)

In the burgeoning age of generative AI, watermarks act as identifiers of provenance and artificial content. We present WAVES (Watermark Analysis Via Enhanced Stress-testing), a benchmark for assessing image watermark robustness, overcoming the limitations of current evaluation methods. WAVES integrates detection and identification tasks and establishes a standardized evaluation protocol comprised of a diverse range of stress tests. The attacks in WAVES range from traditional image distortions to advanced, novel variations of diffusive, and adversarial attacks. Our evaluation examines two pivotal dimensions: the degree of image quality degradation and the efficacy of watermark detection after attacks. Our novel, comprehensive evaluation reveals previously undetected vulnerabilities of several modern watermarking algorithms. We envision WAVES as a toolkit for the future development of robust watermarks. The project is available at https://wavesbench.github.io/