mlsanityData to 7 Oct 2026

Leave No Stone Unturned: Uncovering Holistic Audio-Visual Intrinsic Coherence for Deepfake Detection

arXiv 2603.23960 ↗First posted 25 Mar 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

The rapid progress of generative AI has enabled hyper-realistic audio-visual deepfakes, intensifying threats to personal security and social trust. Most existing deepfake detectors rely either on uni-modal artifacts or audio-visual discrepancies, failing to jointly leverage both sources of information. Moreover, detectors that rely on generator-specific artifacts tend to exhibit degraded generalization when confronted with unseen forgeries. We argue that robust and generalizable detection should be grounded in intrinsic audio-visual coherence within and across modalities. Accordingly, we propose HAVIC, a Holistic Audio-Visual Intrinsic Coherence-based deepfake detector. HAVIC first learns priors of modality-specific structural coherence, inter-modal micro- and macro-coherence by pre-training on authentic videos. Based on the learned priors, HAVIC further performs holistic adaptive aggregation to dynamically fuse audio-visual features for deepfake detection. Additionally, we introduce HiFi-AVDF, a high-fidelity audio-visual deepfake dataset featuring both text-to-video and image-to-video forgeries from state-of-the-art commercial generators. Extensive experiments across several benchmarks demonstrate that HAVIC significantly outperforms existing state-of-the-art methods, achieving improvements of 9.39% AP and 9.37% AUC on the most challenging cross-dataset scenario. Our code and dataset are available at https://github.com/tuffy-studio/HAVIC.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 72 of its 96 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

72 of 96 references.

References · 72

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  9. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  10. 2020Supervised Contrastive Learningcited by 2,803
  11. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  12. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
38 more
  1. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  2. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  3. 2020First Order Motion Model for Image Animationcited by 698
  4. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 669
  5. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  6. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  7. 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 518
  8. 2019Face X-ray for More General Face Forgery Detectioncited by 469
  9. 2022Detecting Deepfakes with Self-Blended Imagescited by 332
  10. 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 319
  11. 2018Deep Audio-Visual Speech Recognitioncited by 318
  12. 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
  13. 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 286
  14. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  15. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  16. 2022Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Predictioncited by 277
  17. 2022Masked Autoencoders that Listencited by 271
  18. 2021Generalizing Face Forgery Detection with High-frequency Featurescited by 261
  19. 2019On the Detection of Digital Face Manipulationcited by 259
  20. 2019FaceShifter: Towards High Fidelity And Occlusion Aware Face Swappingcited by 221
  21. 2021WildDeepfake: A Challenging Real-World Dataset for Deepfake Detectioncited by 207
  22. 2021Audio-Driven Emotional Video Portraitscited by 180
  23. 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 179
  24. 2022Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detectioncited by 172
  25. 2018Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separationcited by 160
  26. 2021Exploring Temporal Coherence for More General Video Face Forgery Detectioncited by 158
  27. 2022Contrastive Audio-Visual Masked Autoencodercited by 142
  28. 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
  29. 2020DeepFaceLab: Integrated, flexible and extensible face-swapping frameworkcited by 129
  30. 2023Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detectioncited by 126
  31. 2016Fast Face-swap Using Convolutional Neural Networkscited by 119
  32. 2018FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Facescited by 118
  33. 2020Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cuescited by 111
  34. 2024Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detectioncited by 102
  35. 2023Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expertcited by 98
  36. 2020Audio-driven Talking Face Video Generation with Learning-based Personalized Head Posecited by 92
  37. 2023Self-Supervised Video Forensics by Audio-Visual Anomaly Detectioncited by 84
  38. 2021KoDF: A Large-scale Korean DeepFake Detection Datasetcited by 80

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.