# Towards a Universal Synthetic Video Detector: From Face or Background Manipulations to Fully AI-Generated Content

arXiv: https://arxiv.org/abs/2412.12278 · HTML: https://mlsanity.com/paper/2412.12278/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 59 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Latte: Latent Diffusion Transformer for Video Generation](https://mlsanity.com/paper/2401.03048/) (2024)
- [VideoCrafter1: Open Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2310.19512/) (2023)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [DIRE for Diffusion-Generated Image Detection](https://mlsanity.com/paper/2303.09295/) (2023)
- [LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models](https://mlsanity.com/paper/2309.15103/) (2023)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2309.15818/) (2023)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [On the detection of synthetic images generated by diffusion models](https://mlsanity.com/paper/2211.00680/) (2022)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping](https://mlsanity.com/paper/1912.13457/) (2019)
- [DeepFakes: a New Threat to Face Recognition? Assessment and Detection](https://mlsanity.com/paper/1812.08685/) (2018)
- [FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals](https://mlsanity.com/paper/1901.02212/) (2019)
- [Intriguing properties of synthetic images: from generative adversarial networks to diffusion models](https://mlsanity.com/paper/2304.06408/) (2023)
- [TALL: Thumbnail Layout for Deepfake Video Detection](https://mlsanity.com/paper/2307.07494/) (2023)
- [ID-Reveal: Identity-aware DeepFake Video Detection](https://mlsanity.com/paper/2012.02512/) (2020)
- [HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping](https://mlsanity.com/paper/2106.09965/) (2021)
- [DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmark](https://mlsanity.com/paper/2405.19707/) (2024)
- [Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Design](https://mlsanity.com/paper/2305.13035/) (2023)
- [LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection](https://mlsanity.com/paper/2401.13856/) (2024)
- [Spatiotemporal Inconsistency Learning for DeepFake Video Detection](https://mlsanity.com/paper/2109.01860/) (2021)

## Built on by (most cited first, 22 of 22)

- [BusterX: MLLM-Powered AI-Generated Video Forgery Detection and Explanation](https://mlsanity.com/paper/2505.12620/) (2025)
- [Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation](https://mlsanity.com/paper/2506.16802/) (2025)
- [DAVID-XR1: Detecting AI-Generated Videos with Explainable Reasoning](https://mlsanity.com/paper/2506.14827/) (2025)
- [RAIDX: A Retrieval-Augmented Generation and GRPO Reinforcement Learning Framework for Explainable Deepfake Detection](https://mlsanity.com/paper/2508.04524/) (2025)
- [Preserving Forgery Artifacts: AI-Generated Video Detection at Native Scale](https://mlsanity.com/paper/2604.04634/) (2026)
- [Beyond Surface Artifacts: Capturing Shared Latent Forgery Knowledge Across Modalities](https://mlsanity.com/paper/2604.07763/) (2026)
- [Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection](https://mlsanity.com/paper/2605.01638/) (2026)
- [Unmasking Puppeteers: Leveraging Biometric Leakage to Expose Impersonation in AI-Based Videoconferencing](https://mlsanity.com/paper/2510.03548/) (2025)
- [SAGA: Source Attribution of Generative AI Videos](https://mlsanity.com/paper/2511.12834/) (2025)
- [CMTA: Leveraging Cross-Modal Temporal Artifacts for Generalizable AI-Generated Video Detection](https://mlsanity.com/paper/2605.00630/) (2026)
- [Grab-3D: Detecting AI-Generated Videos from 3D Geometric Temporal Consistency](https://mlsanity.com/paper/2512.13665/) (2025)
- [EvolveReason: Self-Evolving Reasoning Paradigm for Explainable Deepfake Facial Image Identification](https://mlsanity.com/paper/2603.07515/) (2026)
- [SWIFT: Sliding Window Reconstruction for Few-Shot Training-Free Generated Video Attribution](https://mlsanity.com/paper/2603.08536/) (2026)
- [Video as Natural Augmentation: Towards Unified AI-Generated Image and Video Detection](https://mlsanity.com/paper/2605.21977/) (2026)
- [Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit](https://mlsanity.com/paper/2606.31004/) (2026)
- [Dataset Biases and Shortcut Learning in Motion-Based AI-Generated Video Detection](https://mlsanity.com/paper/2607.00948/) (2026)
- [EA-Swin: An Embedding-Agnostic Swin Transformer for AI-Generated Video Detection](https://mlsanity.com/paper/2602.17260/) (2026)
- [Moir'e Video Authentication: A Physical Signature Against AI Video Generation](https://mlsanity.com/paper/2604.01654/) (2026)
- [Adversarial Deepfake Generation and an Investigation of Purification-Based Adversarial Detection](https://mlsanity.com/paper/2607.25842/) (2026)
- [Retrieval-Driven Training-Free AI-Generated Video Attribution](https://mlsanity.com/paper/2607.28955/) (2026)
- [TUE-Detector: A Tool-Using Expert MLLM-Based Detector for AI-Generated Videos](https://mlsanity.com/paper/2608.30704/) (2026)
- [AI or Real: Detecting Partially Altered Videos Under Resource-Constrained Environments](https://mlsanity.com/paper/2609.20263/) (2026)

## Abstract (arXiv)

Existing DeepFake detection techniques primarily focus on facial manipulations, such as face-swapping or lip-syncing. However, advancements in text-to-video (T2V) and image-to-video (I2V) generative models now allow fully AI-generated synthetic content and seamless background alterations, challenging face-centric detection methods and demanding more versatile approaches. To address this, we introduce the underlineUniversal underlineNetwork for underlineIdentifying underlineTampered and synthunderlineEtic videos (textttUNITE) model, which, unlike traditional detectors, captures full-frame manipulations. textttUNITE extends detection capabilities to scenarios without faces, non-human subjects, and complex background modifications. It leverages a transformer-based architecture that processes domain-agnostic features extracted from videos via the SigLIP-So400M foundation model. Given limited datasets encompassing both facial/background alterations and T2V/I2V content, we integrate task-irrelevant data alongside standard DeepFake datasets in training. We further mitigate the model's tendency to over-focus on faces by incorporating an attention-diversity (AD) loss, which promotes diverse spatial attention across video frames. Combining AD loss with cross-entropy improves detection performance across varied contexts. Comparative evaluations demonstrate that textttUNITE outperforms state-of-the-art detectors on datasets (in cross-data settings) featuring face/background manipulations and fully synthetic T2V/I2V videos, showcasing its adaptability and generalizable detection capabilities.