# WARP: A Unified Benchmark for Invisible Image Watermarking -- Robustness and Protection Against Attacks

arXiv: https://arxiv.org/abs/2609.40031 · HTML: https://mlsanity.com/paper/2609.40031/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 43 of its 88 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 30 of 43)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [HiDDeN: Hiding Data With Deep Networks](https://mlsanity.com/paper/1807.09937/) (2018)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [The Stable Signature: Rooting Watermarks in Latent Diffusion Models](https://mlsanity.com/paper/2303.15435/) (2023)
- [StegaStamp: Invisible Hyperlinks in Physical Photographs](https://mlsanity.com/paper/1904.05343/) (2019)
- [Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data](https://mlsanity.com/paper/2007.08457/) (2020)
- [Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust](https://mlsanity.com/paper/2305.20030/) (2023)
- [A Self-supervised Approach for Adversarial Robustness](https://mlsanity.com/paper/2006.04924/) (2020)
- [Robust Invisible Video Watermarking with Attention](https://mlsanity.com/paper/1909.01285/) (2019)
- [Gaussian Shading: Provable Performance-Lossless Image Watermarking for Diffusion Models](https://mlsanity.com/paper/2404.04956/) (2024)
- [WAVES: Benchmarking the Robustness of Image Watermarks](https://mlsanity.com/paper/2401.08573/) (2024)
- [Adversarial Attacks and Defences Competition](https://mlsanity.com/paper/1804.00097/) (2018)
- [RingID: Rethinking Tree-Ring Watermarking for Enhanced Multi-Key Identification](https://mlsanity.com/paper/2404.14055/) (2024)
- [Towards Blind Watermarking: Combining Invertible and Non-invertible Mechanisms](https://mlsanity.com/paper/2212.12678/) (2022)
- [Leveraging Optimization for Adaptive Attacks on Image Watermarks](https://mlsanity.com/paper/2309.16952/) (2023)
- [TrustMark: Universal Watermarking for Arbitrary Resolution Images](https://mlsanity.com/paper/2311.18297/) (2023)
- [Image Watermarks are Removable Using Controllable Regeneration from Clean Noise](https://mlsanity.com/paper/2410.05470/) (2024)
- [Video Seal: Open and Efficient Video Watermarking](https://mlsanity.com/paper/2412.09492/) (2024)
- [UniEdit-Flow: Unleashing Inversion and Editing in the Era of Flow Models](https://mlsanity.com/paper/2504.13109/) (2025)

## Abstract (arXiv)

Digital image watermarking is increasingly critical in media contexts, as emerging regulations and industry practices require marking AI-generated content and ensuring traceable sources to prevent manipulation or misuse. Recent advances in invisible watermarking methods highlight the need to update existing benchmarking practices to reflect current techniques and evaluation criteria. We address this by introducing WARP -- a unified framework and benchmark for evaluating the robustness of invisible watermarks. WARP incorporates 32 recent classical, deep, and generative watermarking methods, as well as 34 different erasing techniques, ranging from traditional distortions to more sophisticated adversarial, purification, and re-embedding attacks. It provides standardized, reproducible, and easily scalable protocols for evaluating perceptual quality, watermark readability, and attack resilience. Using WARP, we extensively evaluate current invisible watermarking techniques, collecting the largest robustness benchmark in the field. Results identify the most robust approaches under both distortion and adversarial conditions, and reveal consistent relationships between watermarking methods and the attack strategies most effective against them. Our experiments also highlight that some of the watermarking methods considered are highly vulnerable to reembedding, even if they are robust to standard distortions. The code is made available at https://github.com/ispras/wibe.