# GEM-TFL: Bridging Weak and Full Supervision for Forgery Localization through EM-Guided Decomposition and Temporal Refinement

arXiv: https://arxiv.org/abs/2603.05095 · HTML: https://mlsanity.com/paper/2603.05095/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 48 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Soft-NMS -- Improving Object Detection With One Line of Code](https://mlsanity.com/paper/1704.04503/) (2017)
- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [TriDet: Temporal Action Detection with Relative Boundary Modeling](https://mlsanity.com/paper/2303.07347/) (2023)
- [UntrimmedNets for Weakly Supervised Action Recognition and Detection](https://mlsanity.com/paper/1703.03329/) (2017)
- [Self-Supervised Video Forensics by Audio-Visual Anomaly Detection](https://mlsanity.com/paper/2301.01767/) (2023)
- [AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset](https://mlsanity.com/paper/2311.15308/) (2023)
- [AutoLoc: Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1807.08333/) (2018)
- [CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learning](https://mlsanity.com/paper/2103.16392/) (2021)
- [AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection](https://mlsanity.com/paper/2406.02951/) (2024)
- [Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localization](https://mlsanity.com/paper/2305.01979/) (2023)
- [UMMAFormer: A Universal Multimodal-adaptive Transformer Framework for Temporal Forgery Localization](https://mlsanity.com/paper/2308.14395/) (2023)
- [Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning](https://mlsanity.com/paper/2412.00175/) (2024)
- [DiMoDif: Discourse Modality-information Differentiation for Audio-visual Deepfake Detection and Localization](https://mlsanity.com/paper/2411.10193/) (2024)
- [MIS-AVoiDD: Modality Invariant and Specific Representation for Audio-Visual Deepfake Detection](https://mlsanity.com/paper/2310.02234/) (2023)
- [A Multimodal Deviation Perceiving Framework for Weakly-Supervised Temporal Forgery Localization](https://mlsanity.com/paper/2507.16596/) (2025)
- [Bridge the Gap: From Weak to Full Supervision for Temporal Action Localization with PseudoFormer](https://mlsanity.com/paper/2504.14860/) (2025)
- [Weakly-supervised Audio Temporal Forgery Localization via Progressive Audio-language Co-learning Network](https://mlsanity.com/paper/2505.01880/) (2025)
- [Audio-Visual Deepfake Detection With Local Temporal Inconsistencies](https://mlsanity.com/paper/2501.08137/) (2025)
- [A Multi-Stream Fusion Approach with One-Class Learning for Audio-Visual Deepfake Detection](https://mlsanity.com/paper/2406.14176/) (2024)
- [Full-Stage Pseudo Label Quality Enhancement for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2407.08971/) (2024)
- [HOLA: Enhancing Audio-visual Deepfake Detection via Hierarchical Contextual Aggregations and Efficient Pre-training](https://mlsanity.com/paper/2507.22781/) (2025)
- [Weakly Supervised Multimodal Temporal Forgery Localization via Multitask Learning](https://mlsanity.com/paper/2508.02179/) (2025)
- [A Unified Framework for Modality-Agnostic Deepfakes Detection](https://mlsanity.com/paper/2307.14491/) (2023)
- [FTFDNet: Learning to Detect Talking Face Video Manipulation with Tri-Modality Interaction](https://mlsanity.com/paper/2307.03990/) (2023)

## Abstract (arXiv)

Temporal Forgery Localization (TFL) aims to precisely identify manipulated segments within videos or audio streams, providing interpretable evidence for multimedia forensics and security. While most existing TFL methods rely on dense frame-level labels in a fully supervised manner, Weakly Supervised TFL (WS-TFL) reduces labeling cost by learning only from binary video-level labels. However, current WS-TFL approaches suffer from mismatched training and inference objectives, limited supervision from binary labels, gradient blockage caused by non-differentiable top-k aggregation, and the absence of explicit modeling of inter-proposal relationships. To address these issues, we propose GEM-TFL (Graph-based EM-powered Temporal Forgery Localization), a two-phase classification-regression framework that effectively bridges the supervision gap between training and inference. Built upon this foundation, (1) we enhance weak supervision by reformulating binary labels into multi-dimensional latent attributes through an EM-based optimization process; (2) we introduce a training-free temporal consistency refinement that realigns frame-level predictions for smoother temporal dynamics; and (3) we design a graph-based proposal refinement module that models temporal-semantic relationships among proposals for globally consistent confidence estimation. Extensive experiments on benchmark datasets demonstrate that GEM-TFL achieves more accurate and robust temporal forgery localization, substantially narrowing the gap with fully supervised methods.