ModalFidelity: Routing Modalities for Deepfake Detection on a Budget
Not on the map. Why?
Abstract (arXiv)
Deepfakes no longer need to fake a whole video. Generators that read the transcript now alter only the few seconds in which a video's meaning turns, so a forgery hides in a small, unknown fraction of the video. Yet detectors still read every one-second window of both the audio and image streams, spending nearly all of their compute where nothing was altered. We observe that deciding where to look is far cheaper than looking. We present ModalFidelity, a lightweight router that previews each window and decides, before any forensic detector runs, which stream is worth reading, under a hard compute budget it can never exceed. On AV-Deepfake1M, reading at most a fifth of the windows, it is more accurate than gating after the detectors at 15.9x less compute, and retains over 96% of the accuracy of an oracle that knows where every forgery lies.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 29 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
21 of 29 references.
References · 21
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
- 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
- 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
- 2020First Order Motion Model for Image Animationcited by 704
- 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
- 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
- 2016Self-critical Sequence Training for Image Captioningcited by 547
- 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
9 more
- 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 181
- 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 180
- 2022Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentationcited by 117
- 2020Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cuescited by 111
- 2023AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Datasetcited by 80
- 2024Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learningcited by 23
- 2025Deepfake Detection that Generalizes Across Benchmarkscited by 21
- 2024SecureSpectra: Safeguarding Digital Identity from Deep Fake Threats via Intelligent Signaturescited by 4
- 2025PhonemeFake: Redefining Deepfake Realism with Language-Driven Segmental Manipulation and Adaptive Bilevel Detectioncited by 3
Built on by
Not cited yet.