The Calibrated Deepfake Trust Score (CDTS): Competence-Coupled Trust Degradation Across Deepfake Detectors
Not on the map. Why?
Abstract (arXiv)
In moderation, provenance, and verification pipelines a deepfake detector's output probability is read as a degree of trust, so its calibration matters as much as raw accuracy. We reframe deepfake detection as a calibrated, self-auditing trust instrument, the Calibrated Deepfake Trust Score (CDTS), and identify what governs its trustworthiness. Our central finding is a competence-trust coupling with a sharp division: the raw score's miscalibration tracks discriminative competence almost perfectly (r = -0.98, -0.98, -0.95 across two convolutional networks and a CLIP vision transformer), and a calibrator deployed without target labels fails with competence just as tightly (r = -0.98 on the primary detector, for isotonic, Platt, and beta calibrators alike). Given target labels, by contrast, any well-specified calibrator repairs any detector, including inverted ones: in-domain calibratability is not competence-limited, and the trust failure is a distribution-shift phenomenon concentrated exactly on the low-competence generators that motivate deployment. Explanation faithfulness rises and falls on the same competence axis. We reach this conclusion after uncovering, and correcting, a tie-handling degeneracy in the standard equal-mass expected-calibration-error (ECE) estimator that fabricates strong spurious competence-calibration coupling on tie-heavy calibrated scores; the same degeneracy invalidates calibration-equity gaps we previously reported, a caution for fairness auditing. Competence is trackable without labels: batch predictive entropy flags generators with high deployed calibration error at ROC-AUC 0.99, and routing source-batches on label-free competence beats confidence-based routing precisely in the low-competence regimes the coupling identifies, while confidence regains the advantage where competence is high. Trust scoring must be competence-aware; CDTS is the mechanism.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 36 references that are in the map. First posted on arXiv 28 Jun 2026.
Builds on
27 of 36 references.
References · 27
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
- 2018Sanity Checks for Saliency Mapscited by 1,149
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
- 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
- 2017Selective Classification for Deep Neural Networkscited by 799
- 2019Explaining Machine Learning Classifiers through Diverse Counterfactual Explanationscited by 612
- 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 573
15 more
- 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
- 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
- 2021Revisiting the Calibration of Modern Neural Networkscited by 387
- 2019Verified Uncertainty Calibrationcited by 307
- 2023DeepfakeBench: A Comprehensive Benchmark of Deepfake Detectioncited by 162
- 2024DF40: Toward Next-Generation Deepfake Detectioncited by 126
- 2020Mitigating Bias in Calibration Error Estimationcited by 100
- 2025Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024cited by 40
- 2022GBDF: Gender Balanced DeepFake Dataset Towards Fair DeepFake Detectioncited by 32
- 2024AI-Face: A Million-Scale Demographically Annotated AI-Generated Face Dataset and Fairness Benchmarkcited by 28
- 2022Analyzing Fairness in Deepfake Detection With Massively Annotated Databasescited by 13
- 2023An adversarial attack approach for eXplainable AI evaluation on deepfake detection modelscited by 7
- 2023Stratification of uncertainties recalibrated by isotonic regression and its impact on calibration error statisticscited by 6
- 2025Robust and Calibrated Detection of Authentic Multimedia Contentcited by 2
- 2026Human-AI Ensembles Improve Deepfake Detection in Low-to-Medium Quality Videoscited by 1
Built on by
Not cited yet.