# A Multimodal Deviation Perceiving Framework for Weakly-Supervised Temporal Forgery Localization

arXiv: https://arxiv.org/abs/2507.16596 · HTML: https://mlsanity.com/paper/2507.16596/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 42 references that are in the map.
- First posted on arXiv 22 Jul 2025.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [TriDet: Temporal Action Detection with Relative Boundary Modeling](https://mlsanity.com/paper/2303.07347/) (2023)
- [Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization](https://mlsanity.com/paper/2210.14457/) (2022)
- [BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation](https://mlsanity.com/paper/2103.06695/) (2021)
- [Not made for each other- Audio-Visual Dissonance-based Deepfake Detection and Localization](https://mlsanity.com/paper/2005.14405/) (2020)
- [Weakly Supervised Object Localization and Detection: A Survey](https://mlsanity.com/paper/2104.07918/) (2021)
- [AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset](https://mlsanity.com/paper/2311.15308/) (2023)
- [Everything at Once -- Multi-modal Fusion Transformer for Video Retrieval](https://mlsanity.com/paper/2112.04446/) (2021)
- [CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning](https://mlsanity.com/paper/2103.16392/) (2021)
- [Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensics](https://mlsanity.com/paper/2403.14077/) (2024)
- [DeepFake Detection Based on the Discrepancy Between the Face and its Context](https://mlsanity.com/paper/2008.12262/) (2020)
- [Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localization](https://mlsanity.com/paper/2305.01979/) (2023)
- [UMMAFormer: A Universal Multimodal-adaptive Transformer Framework for Temporal Forgery Localization](https://mlsanity.com/paper/2308.14395/) (2023)
- [Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature](https://mlsanity.com/paper/2303.12332/) (2023)
- [MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection](https://mlsanity.com/paper/2307.02733/) (2023)
- [Detection of Deepfake Videos Using Long Distance Attention](https://mlsanity.com/paper/2106.12832/) (2021)
- [Complete Instances Mining for Weakly Supervised Instance Segmentation](https://mlsanity.com/paper/2402.07633/) (2024)
- [Weakly-supervised Audio Temporal Forgery Localization via Progressive Audio-language Co-learning Network](https://mlsanity.com/paper/2505.01880/) (2025)
- [SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints](https://mlsanity.com/paper/2412.09981/) (2024)
- [Probabilistic Vision-Language Representation for Weakly Supervised Temporal Action Localization](https://mlsanity.com/paper/2408.05955/) (2024)

## Built on by (most cited first, 8 of 8)

- [MARE: Multimodal Alignment and Reinforcement for Explainable Deepfake Detection via Vision-Language Models](https://mlsanity.com/paper/2601.20433/) (2026)
- [Weakly Supervised Multimodal Temporal Forgery Localization via Multitask Learning](https://mlsanity.com/paper/2508.02179/) (2025)
- [CIEC: Coupling Implicit and Explicit Cues for Multimodal Weakly Supervised Manipulation Localization](https://mlsanity.com/paper/2602.02175/) (2026)
- [Detecting AI-Generated Video: A Vision-Language Dual-View Survey](https://mlsanity.com/paper/2607.10787/) (2026)
- [I2VShield: An Efficient Proactive Defense Framework against DiT-based Image-to-Video Models](https://mlsanity.com/paper/2607.25522/) (2026)
- [GEM-TFL: Bridging Weak and Full Supervision for Forgery Localization through EM-Guided Decomposition and Temporal Refinement](https://mlsanity.com/paper/2603.05095/) (2026)
- [Word-Anchored Temporal Forgery Localization](https://mlsanity.com/paper/2603.06220/) (2026)
- [HumanForge: A Human-Centric Deepfake Video Benchmark with Multi-Agent Forgery Rationales](https://mlsanity.com/paper/2607.08705/) (2026)

## Abstract (arXiv)

Current researches on Deepfake forensics often treat detection as a classification task or temporal forgery localization problem, which are usually restrictive, time-consuming, and challenging to scale for large datasets. To resolve these issues, we present a multimodal deviation perceiving framework for weakly-supervised temporal forgery localization (MDP), which aims to identify temporal partial forged segments using only video-level annotations. The MDP proposes a novel multimodal interaction mechanism (MI) and an extensible deviation perceiving loss to perceive multimodal deviation, which achieves the refined start and end timestamps localization of forged segments. Specifically, MI introduces a temporal property preserving cross-modal attention to measure the relevance between the visual and audio modalities in the probabilistic embedding space. It could identify the inter-modality deviation and construct comprehensive video features for temporal forgery localization. To explore further temporal deviation for weakly-supervised learning, an extensible deviation perceiving loss has been proposed, aiming at enlarging the deviation of adjacent segments of the forged samples and reducing that of genuine samples. Extensive experiments demonstrate the effectiveness of the proposed framework and achieve comparable results to fully-supervised approaches in several evaluation metrics.