# MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detection

arXiv: https://arxiv.org/abs/2404.08452 · HTML: https://mlsanity.com/paper/2404.08452/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 76 of its 103 references that are in the map.
- First posted on arXiv 12 Apr 2024.

## Placed in

- Mixture-of-experts / [Vision mixture-of-experts](https://mlsanity.com/moe/vision-mixture-of-experts/)

## Builds on (most cited first, 30 of 76)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Mixture-of-Experts with Expert Choice Routing](https://mlsanity.com/paper/2202.09368/) (2022)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts](https://mlsanity.com/paper/2111.02358/) (2021)
- [Detecting Deepfakes with Self-Blended Images](https://mlsanity.com/paper/2204.08376/) (2022)
- [BASE Layers: Simplifying Training of Large, Sparse Models](https://mlsanity.com/paper/2103.16716/) (2021)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning](https://mlsanity.com/paper/2210.08823/) (2022)
- [Generalizing Face Forgery Detection with High-frequency Features](https://mlsanity.com/paper/2103.12376/) (2021)

## Built on by (most cited first, 27 of 27)

- [Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection](https://mlsanity.com/paper/2411.15633/) (2024)
- [Low-Rank Adaptation for Foundation Models: A Comprehensive Review](https://mlsanity.com/paper/2501.00365/) (2024)
- [Fair Deepfake Detectors Can Generalize](https://mlsanity.com/paper/2507.02645/) (2025)
- [Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges](https://mlsanity.com/paper/2406.06965/) (2024)
- [AvatarShield: Visual Reinforcement Learning for Human-Centric Synthetic Video Detection](https://mlsanity.com/paper/2505.15173/) (2025)
- [ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack](https://mlsanity.com/paper/2508.07402/) (2025)
- [EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detection](https://mlsanity.com/paper/2603.17343/) (2026)
- [Benchmarking Unified Face Attack Detection via Hierarchical Prompt Tuning](https://mlsanity.com/paper/2505.13327/) (2025)
- [On the Holistic Approach for Detecting Human Image Forgery](https://mlsanity.com/paper/2601.04715/) (2026)
- [WMamba: Wavelet-based Mamba for Face Forgery Detection](https://mlsanity.com/paper/2501.09617/) (2025)
- [Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Experts](https://mlsanity.com/paper/2505.14088/) (2025)
- [DevFD: Developmental Face Forgery Detection by Learning Shared and Orthogonal LoRA Subspaces](https://mlsanity.com/paper/2509.19230/) (2025)
- [Propose and Rectify: A Forensics-Driven MLLM Framework for Image Manipulation Localization](https://mlsanity.com/paper/2508.17976/) (2025)
- [BIG-MoE: Bypass Isolated Gating MoE for Generalized Multimodal Face Anti-Spoofing](https://mlsanity.com/paper/2412.18065/) (2024)
- [CorrDetail: Visual Detail Enhanced Self-Correction for Face Forgery Detection](https://mlsanity.com/paper/2507.05302/) (2025)
- [Towards Generalizable Deepfake Detection via Forgery-aware Audio-Visual Adaptation: A Variational Bayesian Approach](https://mlsanity.com/paper/2511.19080/) (2025)
- [SAKED: Mitigating Hallucination in Large Vision-Language Models via Stability-Aware Knowledge Enhanced Decoding](https://mlsanity.com/paper/2602.09825/) (2026)
- [FeatDistill: A Feature Distillation Enhanced Multi-Expert Ensemble Framework for Robust AI-generated Image Detection](https://mlsanity.com/paper/2603.21939/) (2026)
- [LOGER: Local--Global Ensemble for Robust Deepfake Detection in the Wild](https://mlsanity.com/paper/2604.03558/) (2026)
- [VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models: Methods and Results](https://mlsanity.com/paper/2509.09190/) (2025)
- [Low-rank Orthogonal Subspace Intervention for Generalizable Face Forgery Detection](https://mlsanity.com/paper/2601.11915/) (2026)
- [StegaFFD: Privacy-Preserving Face Forgery Detection via Fine-Grained Steganographic Domain Lifting](https://mlsanity.com/paper/2603.02886/) (2026)
- [Only Train Once: Uncertainty-Aware One-Class Learning for Face Authenticity Detection](https://mlsanity.com/paper/2605.10040/) (2026)
- [FaceMoE: Mixture of Experts for Low-Resolution Face Recognition](https://mlsanity.com/paper/2606.32040/) (2026)
- [Bridging Adversarial and Collaborative Learning for AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2608.24372/) (2026)
- [Unifying Semantic Priors and High-Frequency Traces: Enhancing V-JEPA with Mixture-of-Experts for Robust Synthetic Image Forensics](https://mlsanity.com/paper/2609.16778/) (2026)
- [DF-CBM: Region-Aware Concept Bottleneck Models for Deepfake Detection](https://mlsanity.com/paper/2609.35096/) (2026)

## Abstract (arXiv)

Deepfakes have recently raised significant trust issues and security concerns among the public. Compared to CNN face forgery detectors, ViT-based methods take advantage of the expressivity of transformers, achieving superior detection performance. However, these approaches still exhibit the following limitations: (1) Fully fine-tuning ViT-based models from ImageNet weights demands substantial computational and storage resources; (2) ViT-based methods struggle to capture local forgery clues, leading to model bias; (3) These methods limit their scope on only one or few face forgery features, resulting in limited generalizability. To tackle these challenges, this work introduces Mixture-of-Experts modules for Face Forgery Detection (MoE-FFD), a generalized yet parameter-efficient ViT-based approach. MoE-FFD only updates lightweight Low-Rank Adaptation (LoRA) and Adapter layers while keeping the ViT backbone frozen, thereby achieving parameter-efficient training. Moreover, MoE-FFD leverages the expressivity of transformers and local priors of CNNs to simultaneously extract global and local forgery clues. Additionally, novel MoE modules are designed to scale the model's capacity and smartly select optimal forgery experts, further enhancing forgery detection performance. Our proposed learning scheme can be seamlessly adapted to various transformer backbones in a plug-and-play manner. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art face forgery detection performance with significantly reduced parameter overhead. The code is released at: https://github.com/LoveSiameseCat/MoE-FFD.