# RobustSora: De-Watermarked Benchmark for Robust AI-Generated Video Detection

arXiv: https://arxiv.org/abs/2512.10248 · HTML: https://mlsanity.com/paper/2512.10248/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Watermarking generated content and models, one of 12 directions in LLM safety and security.
- Builds on 42 of its 66 references that are in the map.
- First posted on arXiv 11 Dec 2025.

## Placed in

- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 30 of 42)

- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020)
- [Towards Universal Fake Image Detectors that Generalize Across Generative Models](https://mlsanity.com/paper/2302.10174/) (2023)
- [DIRE for Diffusion-Generated Image Detection](https://mlsanity.com/paper/2303.09295/) (2023)
- [The Stable Signature: Rooting Watermarks in Latent Diffusion Models](https://mlsanity.com/paper/2303.15435/) (2023)
- [StegaStamp: Invisible Hyperlinks in Physical Photographs](https://mlsanity.com/paper/1904.05343/) (2019)
- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust](https://mlsanity.com/paper/2305.20030/) (2023)
- [Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200k](https://mlsanity.com/paper/2503.09642/) (2025)
- [Exploring Temporal Coherence for More General Video Face Forgery Detection](https://mlsanity.com/paper/2108.06693/) (2021)
- [AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Error](https://mlsanity.com/paper/2401.17879/) (2024)
- [EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protection](https://mlsanity.com/paper/2312.08883/) (2023)
- [SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Model](https://mlsanity.com/paper/2412.04292/) (2024)
- [DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmark](https://mlsanity.com/paper/2405.19707/) (2024)
- [DiffuEraser: A Diffusion Model for Video Inpainting](https://mlsanity.com/paper/2501.10018/) (2025)
- [Vript: A Video Is Worth Thousands of Words](https://mlsanity.com/paper/2406.06040/) (2024)
- [FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistant](https://mlsanity.com/paper/2408.10072/) (2024)
- [UltraVideo: High-Quality UHD Video Dataset with Comprehensive Captions](https://mlsanity.com/paper/2506.13691/) (2025)
- [LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models](https://mlsanity.com/paper/2410.09732/) (2024)
- [Video Seal: Open and Efficient Video Watermarking](https://mlsanity.com/paper/2412.09492/) (2024)
- [DVMark: A Deep Multiscale Framework for Video Watermarking](https://mlsanity.com/paper/2104.12734/) (2021)
- [On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection](https://mlsanity.com/paper/2410.23623/) (2024)
- [X2-DFD: A framework for eXplainable and eXtendable Deepfake Detection](https://mlsanity.com/paper/2410.06126/) (2024)
- [VideoShield: Regulating Diffusion-based Video Generation Models via Watermarking](https://mlsanity.com/paper/2501.14195/) (2025)
- [Ivy-Fake: A Unified Explainable Framework and Benchmark for Image and Video AIGC Detection](https://mlsanity.com/paper/2506.00979/) (2025)

## Built on by (most cited first, 3 of 3)

- [The Watermark Shortcut: How Provenance Marking Sabotages Audio Deepfake Detection](https://mlsanity.com/paper/2606.23335/) (2026)
- [Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit](https://mlsanity.com/paper/2606.31004/) (2026)
- [EA-Swin: An Embedding-Agnostic Swin Transformer for AI-Generated Video Detection](https://mlsanity.com/paper/2602.17260/) (2026)

## Abstract (arXiv)

The proliferation of AI-generated video models poses new challenges to information integrity and digital trust. A key confound, however, remains unaddressed: commercial generators embed visible overlay watermarks for provenance tracking, yet no existing benchmark controls for this variable, leaving open whether detectors learn genuine generation artefacts or merely associate watermark patterns with AI-generated labels. We present RobustSora, a benchmark of 6,500 manually verified videos in four categories: Authentic-Clean (A-C), Generated-Watermarked (G-W), Generated-DeWatermarked (G-DeW), and Authentic-Spoofed (A-S), sourced from Vript, DVF, and UltraVideo (authentic) and from Sora, Sora 2, Pika, Open-Sora 2, and KLing (generated). Two evaluation tasks isolate watermark effects: Task-I (Watermark Erasure Robustness) tests detection on watermark-removed AI videos; Task-II (Watermark Spoofing Robustness) measures false-alarm rates on authentic videos injected with fake watermarks. Across ten models spanning specialized detectors, transformer classifiers, and MLLMs, watermark manipulation induces accuracy changes of -9.4 to +1.6 pp (mean 6.6 pp; p<0.01 for 7/10 models on each task). A placebo control bounds inpainting-artefact confounds at ≤2 pp, and a watermark-aware training augmentation recovers 3-4 pp on both tasks, together providing causal evidence that detectors actively rely on watermark cues. Per-generator breakdown shows that Sora 2 induces drops of -11 to -14 pp versus -3 to -6 pp for Pika and Open-Sora 2, indicating that watermark prominence, rather than detector architecture, is the principal driver of dependency. These results argue for watermark-aware evaluation and training in AIGC video detection. Dataset, evaluation code, and pretrained checkpoints will be released.