mlsanityData to 14 Oct 2026

DF-MoE: Generalizable Deepfake Detection via Multimodal Sparse Mixture-of-Experts

arXiv 2608.23363 ↗First posted 24 Aug 2026cs.CV cs.AI cs.LGcited by 0

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

Audio-visual deepfake detection is an actively studied topic, where one of the main challenges is to develop detectors able to generalize across deepfake generation methods. We conjecture that overfitting can be mitigated by extracting multiple high-level cues from the available audio and visual modalities via pre-trained models. We therefore assemble a wide variety of pre-trained models to extract features that encode mouth movements, face parsing, facial expressions, head pose, gaze tracking, heart rate, audio emotion and speech activity. We further integrate both unimodal and multimodal cues via a Mixture-of-Experts (MoE) backbone to detect deepfakes. We perform in-domain and cross-domain experiments on five benchmarks for deepfake detection (MAVOS-DD, AVLips, PolyGlotFake, BioDeepAV, FakeAVCeleb) to compare our framework (DF-MoE) with state-of-the-art methods. Our results indicate that DF-MoE obtains superior deepfake detection results, surpassing all competing methods. We release our code at https://github.com/vladhondru25/DF-MoE.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 91 of its 119 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

91 of 119 references.

References · 91

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  8. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  9. 2025DINOv3cited by 1,414
  10. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
  11. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
  12. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
38 more
  1. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  2. 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 575
  3. 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 523
  4. 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 425
  5. 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
  6. 2021Multi-attentional Deepfake Detectioncited by 341
  7. 2022Detecting Deepfakes with Self-Blended Imagescited by 338
  8. 2018Exposing DeepFake Videos By Detecting Face Warping Artifactscited by 319
  9. 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 287
  10. 2019On the Detection of Digital Face Manipulationcited by 260
  11. 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 206
  12. 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
  13. 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 181
  14. 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 180
  15. 2023UCF: Uncovering Common Features for Generalizable Deepfake Detectioncited by 180
  16. 2020Learning Self-Consistency for Deepfake Detectioncited by 179
  17. 2022Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detectioncited by 173
  18. 2020End-to-end anti-spoofing with RawNet2cited by 164
  19. 2017Fine-Grained Head Pose Estimation Without Keypointscited by 163
  20. 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
  21. 2023Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detectioncited by 127
  22. 2024AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Errorcited by 125
  23. 2022Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalizationcited by 123
  24. 2019Recurrent Convolutional Strategies for Face Manipulation Detection in Videoscited by 118
  25. 2021M2TR: Multi-modal Multi-scale Transformers for Deepfake Detectioncited by 114
  26. 2023TALL: Thumbnail Layout for Deepfake Video Detectioncited by 100
  27. 2017Smart Mining for Deep Metric Learningcited by 96
  28. 2022Protecting Celebrities from DeepFake with Identity Consistency Transformercited by 95
  29. 2021HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swappingcited by 92
  30. 2024LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detectioncited by 89
  31. 2023Self-Supervised Video Forensics by Audio-Visual Anomaly Detectioncited by 85
  32. 2022CORE: Consistent Representation Learning for Face Forgery Detectioncited by 74
  33. 2024Sonic: Shifting Focus to Global Audio Perception in Portrait Animationcited by 72
  34. 2024Preserving Fairness Generalization in Deepfake Detectioncited by 70
  35. 2025Let Them Talk: Audio-Driven Multi-Person Conversational Video Generationcited by 66
  36. 2021BiHPF: Bilateral High-Pass Filters for Robust Deepfake Detectioncited by 60
  37. 2024AVFF: Audio-Visual Feature Fusion for Video Deepfake Detectioncited by 59
  38. 2024Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuningcited by 59

50 most cited of 91 All 91 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Leveraging large multimodal models for audio-video deepfake detection: a pilot study
  2. 2025PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis
  3. 2024Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges
  4. 2025Mixture of Low-Rank Adapter Experts in Generalizable Audio Deepfake Detection
  5. 2026ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection
  6. 2025HOLA: Enhancing Audio-visual Deepfake Detection via Hierarchical Contextual Aggregations and Efficient Pre-training
  7. 2026Teacher-Student Structure for Domain Adaptation in Ensemble Audio-Visual Video Deepfake Detection
  8. 2024Deepfake Generation and Detection: A Benchmark and Survey