Architecture-Adaptive Uncertainty Fusion for Deepfake Detection
Not on the map. Why?
Abstract (arXiv)
Deepfake detection systems achieve near-perfect accuracy on benchmarks, yet forensic deployment demands reliable prediction uncertainty. Existing uncertainty quantification (UQ) methods rely on single sources and ignore that optimal uncertainty composition varies across architectures. We propose Correlation-Optimized Fusion (COF), an architecture-adaptive framework that fuses five complementary uncertainty sources -- epistemic, aleatoric, calibration, conformal, and distributional -- by maximizing Pearson correlation between fused uncertainty scores and prediction errors via constrained optimization on the probability simplex. COF requires no model modifications and only 42 s of weight optimization, compared to 20--45 h for a 5-model Deep Ensemble. Evaluation across eleven architectures on FaceForensics++ reveals a fundamental trade-off: under matched train/evaluation protocol, non-linear methods achieve approximately 5--6% higher in-domain correlation than COF (mean r = 0.438), but this reverses under distribution shift. On CelebDF, COF outperforms Random Forest in 9/11 architectures with up to 7.3x higher correlation (MaxViT-B: r = 0.249 vs. 0.034); RF degrades 85% cross-domain to r = 0.071, whereas COF retains substantially more signal (74% drop to r = 0.116). Cross-dataset evaluation on CelebDF and DFDC reveals catastrophic generalization failure across all methods: in-domain correlations of 0.41--0.47 collapse to near-zero externally (mean degradation 90.7%), with seven of eleven architectures exhibiting uncertainty inversion. These results establish COF as a practical, interpretable framework for controlled-distribution deployment and identify domain-adaptive UQ as the central open challenge for forensic deployment.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 44 references that are in the map. First posted on arXiv 4 Jun 2026.
Builds on
34 of 44 references.
References · 34
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2022A ConvNet for the 2020scited by 3,579
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
- 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
22 more
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
- 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
- 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
- 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 732
- 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 575
- 2015Weight Uncertainty in Neural Networkscited by 554
- 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 523
- 2022MaxViT: Multi-Axis Vision Transformercited by 476
- 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
- 2020Classification with Valid and Adaptive Coveragecited by 396
- 2020Calibrating Deep Neural Networks using Focal Losscited by 357
- 2019Deep Evidential Regressioncited by 355
- 2021Combining EfficientNet and Vision Transformers for Video Deepfake Detectioncited by 82
- 2021Deepfake Video Detection Using Convolutional Vision Transformercited by 79
- 2025Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024cited by 45
- 2022MINTIME: Multi-Identity Size-Invariant Video Deepfake Detectioncited by 23
- 2023DomainForensics: Exposing Face Forgery across Domains via Bi-directional Adaptationcited by 8
- 2025Think Twice before Adaptation: Improving Adaptability of DeepFake Detection via Online Test-Time Adaptationcited by 1
- 2025Towards Generalizable Deepfake Detection via Forgery-aware Audio-Visual Adaptation: A Variational Bayesian Approachcited by 1
Built on by
Not cited yet.