# The Calibrated Deepfake Trust Score (CDTS): Competence-Coupled Trust Degradation Across Deepfake Detectors

arXiv: https://arxiv.org/abs/2606.29484 · HTML: https://mlsanity.com/paper/2606.29484/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 28 Jun 2026.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Revisiting the Calibration of Modern Neural Networks](https://mlsanity.com/paper/2106.07998/) (2021)
- [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019)
- [DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection](https://mlsanity.com/paper/2307.01426/) (2023)
- [DF40: Toward Next-Generation Deepfake Detection](https://mlsanity.com/paper/2406.13495/) (2024)
- [Mitigating Bias in Calibration Error Estimation](https://mlsanity.com/paper/2012.08668/) (2020)
- [Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024](https://mlsanity.com/paper/2503.02857/) (2025)
- [GBDF: Gender Balanced DeepFake Dataset Towards Fair DeepFake Detection](https://mlsanity.com/paper/2207.10246/) (2022)
- [AI-Face: A Million-Scale Demographically Annotated AI-Generated Face Dataset and Fairness Benchmark](https://mlsanity.com/paper/2406.00783/) (2024)
- [Analyzing Fairness in Deepfake Detection With Massively Annotated Databases](https://mlsanity.com/paper/2208.05845/) (2022)
- [An adversarial attack approach for eXplainable AI evaluation on deepfake detection models](https://mlsanity.com/paper/2312.06627/) (2023)
- [Stratification of uncertainties recalibrated by isotonic regression and its impact on calibration error statistics](https://mlsanity.com/paper/2306.05180/) (2023)
- [Robust and Calibrated Detection of Authentic Multimedia Content](https://mlsanity.com/paper/2512.15182/) (2025)
- [Human-AI Ensembles Improve Deepfake Detection in Low-to-Medium Quality Videos](https://mlsanity.com/paper/2603.14658/) (2026)

## Abstract (arXiv)

In moderation, provenance, and verification pipelines a deepfake detector's output probability is read as a degree of trust, so its calibration matters as much as raw accuracy. We reframe deepfake detection as a calibrated, self-auditing trust instrument, the Calibrated Deepfake Trust Score (CDTS), and identify what governs its trustworthiness. Our central finding is a competence-trust coupling with a sharp division: the raw score's miscalibration tracks discriminative competence almost perfectly (r = -0.98, -0.98, -0.95 across two convolutional networks and a CLIP vision transformer), and a calibrator deployed without target labels fails with competence just as tightly (r = -0.98 on the primary detector, for isotonic, Platt, and beta calibrators alike). Given target labels, by contrast, any well-specified calibrator repairs any detector, including inverted ones: in-domain calibratability is not competence-limited, and the trust failure is a distribution-shift phenomenon concentrated exactly on the low-competence generators that motivate deployment. Explanation faithfulness rises and falls on the same competence axis. We reach this conclusion after uncovering, and correcting, a tie-handling degeneracy in the standard equal-mass expected-calibration-error (ECE) estimator that fabricates strong spurious competence-calibration coupling on tie-heavy calibrated scores; the same degeneracy invalidates calibration-equity gaps we previously reported, a caution for fairness auditing. Competence is trackable without labels: batch predictive entropy flags generators with high deployed calibration error at ROC-AUC 0.99, and routing source-batches on label-free competence beats confidence-based routing precisely in the low-competence regimes the coupling identifies, while confidence regains the advantage where competence is high. Trust scoring must be competence-aware; CDTS is the mechanism.