# Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit

arXiv: https://arxiv.org/abs/2606.31004 · HTML: https://mlsanity.com/paper/2606.31004/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 49 references that are in the map.
- First posted on arXiv 30 Jun 2026.

## Builds on (most cited first, 30 of 35)

- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators](https://mlsanity.com/paper/2303.13439/) (2023)
- [VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models](https://mlsanity.com/paper/2401.09047/) (2024)
- [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020)
- [VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Models](https://mlsanity.com/paper/2403.06098/) (2024)
- [Compressed Video Action Recognition](https://mlsanity.com/paper/1712.00636/) (2017)
- [DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmark](https://mlsanity.com/paper/2405.19707/) (2024)
- [Croissant: A Metadata Format for ML-Ready Datasets](https://mlsanity.com/paper/2403.19546/) (2024)
- [Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videos](https://mlsanity.com/paper/2407.16124/) (2024)
- [On the Content Bias in Fr'echet Video Distance](https://mlsanity.com/paper/2404.12391/) (2024)
- [Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024](https://mlsanity.com/paper/2503.02857/) (2025)
- [You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos](https://mlsanity.com/paper/2303.07863/) (2023)
- [DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition](https://mlsanity.com/paper/1901.03460/) (2019)
- [AI-Generated Video Detection via Perceptual Straightening](https://mlsanity.com/paper/2507.00583/) (2025)
- [D3: Training-Free AI-Generated Video Detection Using Second-Order Features](https://mlsanity.com/paper/2508.00701/) (2025)
- [Physics-Driven Spatiotemporal Modeling for AI-Generated Video Detection](https://mlsanity.com/paper/2510.08073/) (2025)
- [Detecting AI-Generated Video via Frame Consistency](https://mlsanity.com/paper/2402.02085/) (2024)
- [Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation](https://mlsanity.com/paper/2506.16802/) (2025)
- [Towards a Universal Synthetic Video Detector: From Face or Background Manipulations to Fully AI-Generated Content](https://mlsanity.com/paper/2412.12278/) (2024)
- [CPGA: Coding Priors-Guided Aggregation Network for Compressed Video Quality Enhancement](https://mlsanity.com/paper/2403.10362/) (2024)
- [How Far are AI-generated Videos from Simulating the 3D Visual World: A Learned 3D Evaluation Approach](https://mlsanity.com/paper/2406.19568/) (2024)
- [VidGuard-R1: AI-Generated Video Detection and Explanation via Reasoning MLLMs and RL](https://mlsanity.com/paper/2510.02282/) (2025)
- [Your One-Stop Solution for AI-Generated Video Detection](https://mlsanity.com/paper/2601.11035/) (2026)
- [VideoVeritas: AI-Generated Video Detection via Perception Pretext Reinforcement Learning](https://mlsanity.com/paper/2602.08828/) (2026)

## Built on by (most cited first, 1 of 1)

- [Detect Early, Escalate Rarely: Anytime Detection of AI-Generated Video from the Compressed Bitstream](https://mlsanity.com/paper/2607.19476/) (2026)

## Abstract (arXiv)

AI-generated video detection benchmarks such as GenVidBench and AIGVDBench are the de facto leaderboards, yet most evaluation protocols leave uncontrolled confounds that can inflate reported generalization. As an existence proof, a three-feature clip-length classifier reaches a leave-one-generator-out (LOGO) AUC of 0.998 on GenVidBench under unaudited evaluation, while measuring nothing about motion. A 20-paper survey finds none applying all six standard controls that would catch this, so we combine them into an audited protocol and apply it to six representative feature sources (three published detectors and three repurposed signal sources), re-running it cross-dataset on AIGVDBench. The audit both debunks and certifies: the trivial classifier collapses to near chance (0.529), a CLIP baseline is caught carrying dataset identity, and the 2025 forensic detector WaveRep clears the floor at out-of-distribution LOGO AUC 0.996 with chance-level real-vs-real coherence. At a deployable FPR of 0.1%, multiple high-AUC methods fall to single-digit recall and the leaderboard order changes, so we recommend an audited tuple (AUC, above-floor margin, operating-point recall, and calibration) over a single number. As a white-box positive control, we add TemporalSpec (codec motion vectors); via cross-substrate feature fusion (XSFF), a second substrate adds genuine complementarity that survives the audit. We release VidAudit, to our knowledge the largest unified and audited detector collection for this task, providing 14 detectors behind one plugin API, a leaderboard, and Croissant metadata, available at https://github.com/KurbanIntelligenceLab/vidaudit. Together, the protocol and toolkit move evaluation from leaderboard rank toward whether a result measures what it claims.