Leave No Stone Unturned: Uncovering Holistic Audio-Visual Intrinsic Coherence for Deepfake Detection
Not on the map. Why?
Abstract (arXiv)
The rapid progress of generative AI has enabled hyper-realistic audio-visual deepfakes, intensifying threats to personal security and social trust. Most existing deepfake detectors rely either on uni-modal artifacts or audio-visual discrepancies, failing to jointly leverage both sources of information. Moreover, detectors that rely on generator-specific artifacts tend to exhibit degraded generalization when confronted with unseen forgeries. We argue that robust and generalizable detection should be grounded in intrinsic audio-visual coherence within and across modalities. Accordingly, we propose HAVIC, a Holistic Audio-Visual Intrinsic Coherence-based deepfake detector. HAVIC first learns priors of modality-specific structural coherence, inter-modal micro- and macro-coherence by pre-training on authentic videos. Based on the learned priors, HAVIC further performs holistic adaptive aggregation to dynamically fuse audio-visual features for deepfake detection. Additionally, we introduce HiFi-AVDF, a high-fidelity audio-visual deepfake dataset featuring both text-to-video and image-to-video forgeries from state-of-the-art commercial generators. Extensive experiments across several benchmarks demonstrate that HAVIC significantly outperforms existing state-of-the-art methods, achieving improvements of 9.39% AP and 9.37% AUC on the most challenging cross-dataset scenario. Our code and dataset are available at https://github.com/tuffy-studio/HAVIC.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 72 of its 96 references that are in the map. First posted on arXiv 25 Mar 2026.
Builds on
72 of 96 references.
References · 72
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2020Supervised Contrastive Learningcited by 2,803
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
38 more
- 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
- 2020First Order Motion Model for Image Animationcited by 698
- 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 669
- 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
- 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 518
- 2019Face X-ray for More General Face Forgery Detectioncited by 469
- 2022Detecting Deepfakes with Self-Blended Imagescited by 332
- 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 319
- 2018Deep Audio-Visual Speech Recognitioncited by 318
- 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
- 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 286
- 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
- 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
- 2022Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Predictioncited by 277
- 2022Masked Autoencoders that Listencited by 271
- 2021Generalizing Face Forgery Detection with High-frequency Featurescited by 261
- 2019On the Detection of Digital Face Manipulationcited by 259
- 2019FaceShifter: Towards High Fidelity And Occlusion Aware Face Swappingcited by 221
- 2021WildDeepfake: A Challenging Real-World Dataset for Deepfake Detectioncited by 207
- 2021Audio-Driven Emotional Video Portraitscited by 180
- 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 179
- 2022Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detectioncited by 172
- 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 160
- 2021Exploring Temporal Coherence for More General Video Face Forgery Detectioncited by 158
- 2022Contrastive Audio-Visual Masked Autoencodercited by 142
- 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
- 2020DeepFaceLab: Integrated, flexible and extensible face-swapping frameworkcited by 129
- 2023Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detectioncited by 126
- 2016Fast Face-swap Using Convolutional Neural Networkscited by 119
- 2018FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Facescited by 118
- 2020Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cuescited by 111
- 2024Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detectioncited by 102
- 2023Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expertcited by 98
- 2020Audio-driven Talking Face Video Generation with Learning-based Personalized Head Posecited by 92
- 2023Self-Supervised Video Forensics by Audio-Visual Anomaly Detectioncited by 84
- 2021KoDF: A Large-scale Korean DeepFake Detection Datasetcited by 80
50 most cited of 72 All 72 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.